trspoft  Ns.,*  s9 


IA  FOR  EVALUATING  THE  EFFECTIVENESS  OF 
1 8RARY  OPERATIONS  AND  SERVICES 

II:  DATA  GATHERING  AND  EVALUATION 


ny|psTTjfPnn  nm<-~ 
!j|  0CT|  0  1363  i;i 

Otekts  ffidHj 


Fire*  .|6*pe?~-t  on  PHASE  |[  of  Contract  DA-2S01 7-AMC- 
2  *ZZ  ' * ,  for  PicotJnny  Ar**nol,  Dover,  New  Jtonwy 


Distribution  of  this  document  I*  unlimited 


John  I .  Thompeon  &  Company 
1U8  22nd  Street,  N,W, 
WasHSn^on,  D X.,  2C$37 


ilUsi  N  ...  M  ■.  •  o  s  ’ 


ATLIS  Report  No.  19 


Criteria  for  Evaluating  The  Effectiveness  of 
Library  Operations  and  Services 


PHASE  II: 
By: 


Date: 


Data  Gathering  and  Evaluation 
C.J.  Vessel, 

K.L.  Moore 
and 

B.A.  Cohrssen 
August  1968 


Final  Report  on  PHASE  II  of  Contract  DA-28017-AMC- 
3483(A)  for  Picatlnny  Arsenal,  Dover,  New  Jersey 


Distribution  of  this  document  is  unlimited 


John  I .  Thompson  &  Company 
1118  22nd  Street,  N.W. 
Washington,  D.C.  20037 


ABSTRACT 


Thi8  report  summarizes  (1)  data  and  information  collected  to  facil- 
tate  the  development  of  criteria  for  the  evaluation  of  the  efficiency  and 
ef fec<-tvene«i8  of  Army  Technical  Libraries;  (2)  the  findings  or*  the  mission 
and  objectives  of  Army  Technical  Libraries;  (3)  the  areas  in  which  adequate 
standards  for  performance  are  feasible,  and  (4)  presents  the  tentative 
(candidate)  criteria  and  proposed  management  techniques  useful  in  implement¬ 
ing  them. 

The  criteria  apply  to  4  general  aspects  of  library  performance : 

(1)  Philosophical  -  criteria  which  relate  to  the  enunciation 
of  the  reasons  for  the  existence  of  the  library, 

(2)  Management  -  criteria  which  -elate  to  the  influence  exerted 
on  the  efficiency  and  effectiveness  of  the  library  by 

(3)  Services  and  products  -  criteria  which  relate  to  the  outputs 

of  the  library  and  measure  of  service  or  nroduct  effectiveness. 

(4)  Operations  -  criteria  which  are  potentially  useful  ss  a  basis 
for  developing  adequate  standards  for  performance  evaluation 
of  the  staff's  professional  actions. 


PREFACE 


It  is  the  end  purpose  of  Contract  DA-28017-AMC~3483(A)  to  establish 
criteria  for  evaluating  Army  Technical  Library  operations  and  services. 
The  criteria  are  intended  to  cover  the  evaluation  of  all  aspects  of  tech¬ 
nical  library  operations  and  services,  including  but  not  limited  to: 


ft. 

Library  staff 

f . 

Service  effectiveness 

b. 

Personnel  utilization 

R  • 

Operations  efficiency 

c . 

Placement  of  library  in 

h. 

Budget  allocation 

organization  it  serves 

1. 

Procurement  and 

d. 

e. 

Space  and  equipment  requirements 
Collections 

maintenance 

Because  the  criteria  must  be  concepts  usable  for  achieving  the  objec¬ 
tives  of  the  study  ard  will  be  applied  pragmatically ,  the  study  must: 

a.  define  each  objective; 

b.  provide  a  method  of  testing  the  criterion  wirh  the  purpose  of 
determining  library  effectiveness. 

The  study  is  divided  into  three  phases: 

Phase  I  -  State-of-the-arL:  conducting  a  literature  search 

reflecting  the  current  state-of-the-art,  covering: 

a.  library  standards; 

b.  methods  of  evaluating  libraries; 

c.  data  and  information  relating  to  the 
library  performance  evaluation. 

Phase  II  -  Data  gathering  and  evaluation. 

Phase  III  -  Establishing  criteria. 

To  introduce  the  reader  to  the  work  and  findings  of  Phase  II,  a  brief 
review  of  Phase  I  may  prove  helpful.  During  Phase  I  a  detail'd  search  was 
made  of  the  literature  of  library  and  management  ’’science"  to  assess  the 
‘state-of-the-art"  of  criteria  and  methods  of  evaluating  the  efficiency  and 
effectiveness  of  library  operations  and  services.  More  than  one  thousand 
rei.«rences  were  located,  and  the  documents  were  reviewed  and  evaluated. 

Those  judged  as  moat  pertinent  and  contributory  to  the  study  were  abstracted 
for  inclusion  in  the  Phase  I  report.  Supplemental  documents  were  included 
as  references.  Definitions  of  such  key  terms  as  "effectiveness,"  "efficiency," 
"criteria,"  "operations"  and  "services"  were  given  and  a  glossary  of  manage¬ 
ment  science  terms  was  provided.  A  matrix  was  constructed  depicting  the 
possible  application  of  various  management  techniques  to  specific  library 
operations  and  services.  A  discussion  of  the  findings  of  the  investigation 
was  presented.  Generally,  the  existing  criteria  are  deemed  i..compWe;  onlv 
a  few  of  them  are  useful  as  a  basis  for  measuring  librarv  efficiency  and 


effectiveness.  The  quantitative  measurement  of  the  value  of  a  library 
service  or  product  such  as  a  literature  search,  bibliography  or  current 
awareness  service  appears  be  difficult  to  assess. 

Selected  suggestions  were  ;cade  for  possible  approaches  to  develop- 
■cat  of  suitable  criteria.  Amonfc  these  was  the  suggestion  that  a 
specific  library  criteria  might  be  developed  by  analyzing  to  what  extent 
the  library  carries  out  its  mission. 

This  report  discusses  the  investigations  and  findings  of  Phase  II. 

The  contractor  was  charged  to  collect  and  evaluate  the  data  on  technical 
libraries  already  available  from  surveys  conducted  by  DoD,  Department  of 
the  Army  and  others.  The  contractor  was  directed  to  visit  a  selected 
number  of  Army  Technical  Libraries  to  gather  additional  information  on 
performance  standards,  work  measurement,  standard  operating  procedures  and 
related  iteiaa.- 

AI the ugh  the  contract  states  that  it  is  the  cask  of  Phase  III  to 
establish,  test  and  evaluate  the  criteria,  the  dividing  line  between  Phase  II 
and  III  is  neither  fixed  nor  precise.  The  very  nature  of  the  work  is  such 
that  the  for®  and  substance  of  the  criteria  begin  to  take  shape  in  Phase  II. 
Thus,  although  this  report  will  not  actually  state  or  recommend  final 
criteria,  a  task  that  will  be  treated  in  Phase  III,  the  analysis  and  evalua¬ 
tion  of  th*  data  and  information  identified  in  Phase  I  and  II  will  suggest 
to  the  readers  the  characteristics  intended  to  describe  these  criteria. 

Phase  II  is  not  intended  to  be  instructional  ir.  the  application  of  tentative 
criteria  or  methods  of  implementation.  Phase  IT  will  present  and  validate  the 
the  criteria  and  methods  of  implementation  suf«,ested  in  this  phase  and  will 
also  present  rletai’ed  inat ructions  on  how  to  tpply  the  tools.  Phase  III 
will  also  present  ranks  for  the  criteria  and  tools  according  to  their 
relative  usefulness  at  various  types  of  libraries. 
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IX 


I.  CANDIDATE  CRITERIA  AND  CONCLUSIONS 


The  statements  in  this  chapter  represent  the  auaurizatioa  of  the 
findings  of  Phase  II  work.  One  would  normally  expect  to  find  them  at 
the  end  of  the  report.  They  are  being  presented  at  this  point,  however, 
so  that  the  reader  may  better  understand  the  developments  in  later  chapters. 

In  reviewing  the  work  of  Phase  II  it  became  apparent  that  the  can¬ 
didate  criteria  fall  into  four  general  categories,  depending  upon  the 
particular  parameter  or  aspect  of  the  library  to  be  evaluated.  These 
classes  of  criteria  are: 

(1)  Philosophical  -  criteria  which  relate  to  the  enunciation  of 
the  reasons  for  the  existence  of  the  library  and  the  purposes 
it  serves.  The  criterion  here  might  be  "the  adequacy  and 
clarity  of  the  library  mission  statement  is  a  criterion  of 
the  effectiveness  of  the  library,"  for  on  it  hinges  the 
ability  of  the  library  staff  to  fulfill  the  purposes  of  the 
organization.  It  is  not  claimed  that  this  class  of  criteria 
would  lead  to  standards  by  which  to  "measure"  in  the  usual 
physical  sense  of  the  term. 

(2)  Management  -  criteria  which  relate  to  the  Influence  exerted 

on  the  effectiveness  of  the  library  by  the  management  practices. 
Here  we  are  suggesting  that  the  effectiveness  of  the  library  is 
a  function  of  how  expertly  the  librarian  (and  other  library 
supervisory  staff  in  the  larger  library)  chooses  his  staff, 
organizes  them  and  directs  their  efforts  towards  producing 
the  services  which  are  essential  to  fulfilling  the  goals  and 
objectivee  of  the  library. 

(3)  Services  and  Producta  -  criteria  which  relate  to  the  outputs 
of  the  library,  that  is,  the  actual  services  performed  for  a 
client  or  the  products  produced  for  the  client's  use.  This 
class  of  criteria  would  relate  to  identification  and  measure¬ 
ment  of  the  characteristics  of  the  services  and  products  and 
also  the  value  of  these  outputs  to  the  clients.  These  criteria 
would  strive  towards  evaluating  the  effectiveness  of  services 
and  products  in  fulfilling  either  individual  client's  needs, 

or  statistically  fulfilling  the  needs  of  populations  of  clients. 

(A)  Operations  -  criteria  which  relate  to  the  more  or  less  routine, 
but  nonetheless  professional  actions  which  are  carried  out 
daily  by  the  staff  in  operating  a  library,  but  which  actually 
determine  the  quality,  quantity,  usefulness  and  cost  of  the 
services  and  products. 


Preceding  Page  Blank 


PHILOSPHICAL 


Criteria 


The  effectiveness  of  an 
Army  Technical  Library  is  a 
function  of  the  extent  to 
vhich  it  supports  the  Mission 
of  the  parent  organization. 

(a'  The  scission  of  the 
A  nay  Technical  Library  is  a 
derivative  of  the  mission  of 
the  parent  organization. 

(b)  The  effectiveness  of  an 
A  nay  Technical  Library  is  a 
function  of  the  adequacy  and 
clarity  of  its  mission  state¬ 
ment  In  enumerating  concrete 
goals  and  objectives,  as  well 
as  specific  library  services 
and  products. 


The  effectiveness  of  an 
Army  Technical  Library  is 
«  function  of  the  closeness 
of  its  affiliation  with  the 
adalnlsfrative  level  re¬ 
sponsible  for  the  organ¬ 
izational  divisions  served 
by  the  library.  The  mission 
of  the  Army  Technical  Library 
in  approved  at  this  admin¬ 
istrative  level  and  im¬ 
plemented  by  the  library 
administrator. 


Method  of  Implementation 


1,  Organization  analysis  -  study 

of  the  purpose  for  the  existence 
of  an  organization,  ita  primary 
and  secondary  functions  and  the 
extent  to  which  it  accomplishes 
its  functions  (see  XXVI  in 
glossary) . 

2.  Research  ~  study  of  missions 
statements  at  other  libraries 
with  similar  types  of  clientele 
(see  XXVII  in  glossary). 

3.  Hunan  alations  -  studies  in 
customer  oriented  planning  and 
control  (see  XVII  in  glossary) . 

4,  P.P.B.S.  -  Planning  -  Program¬ 
ming  -  Budgeting  System  aids 
program  directors  in  planning, 
delineating  objectives  and 
analyzing  costs  and  benefits  and 
in  budgeting  to  maximize  benefits 
(see  V  in  glossary)  . 


1,  Organization  analysis-utilizing 
organization  charts  to  identify 
relationships  between  line  and 
staff  units  within  a"  organiza¬ 
tion  and  to  delineate  areas  of 
responsibility  of  each  unit. 

Also  provides  a  view  of  the  dis¬ 
tribution  of  responsibilities 
making  recognition  of  role  in¬ 
consistencies  with  organizational 
goals  easier  (see  XXVI  and  IX  in 
glossary) . 


MANAGEMENT 


Criteria 


The  effectiveness  of  an 


Method  of  Implementation 


Army  Technical  Library  is  a 
function  of  the  extent  to 
which  the  library  admin¬ 
istrator  manages  his  resources 
to  provide  the  combination 
of  services  and  products  which 
give  optimum  support  to  the 
library  mission,  goals  and 
objectives . 


Cost-effectiveness  analysis 
and  P.P.B.S.  -  utilizes  a 
"systems  approach"  to  quantify 
costs  and  effectiveness  through 
system  simulation  models.  The 
system  model  facilitates  math¬ 
ematical  modeling  which  can  be 
the  basis  for  planning,  program 
aing  and  budgeting  decisions 
(see  II  and  V  in  glossary). 


(a)  The  ef fectiveness  of  an 
Army  Technical  Library  is  a 
r unction  of  the  extent  to 
which  the  librarian  meets  quantity 
requirements  fcr  products  and 
services . 


(b)  The  effectiveness  of  an 
Army  Technical  Library  is  a 
function  of  the  extent  to 
which  the  librarian  meets  quality 
requirements  for  services  and  , 
products.  | 


2.  Utility  analysis  -  utilizes 
managers’  judgments  as  to  the 
value  of  services  and  operations 
in  supporting  the  library  mis¬ 
sion.  Basic  judgments  of  the 
relative  values  of  each  service 
and  operation  are  recorded  in 
such  a  way  that  mathematical 
equations  can  be  used  to  unify 
these  basic  judgments  and  to 
resolve  the  balance  between 
operations  which  gives  optimum 
support  tc  the  library  mission, 
goals  a-id  objectives  (see  IV 
in  glossary) . 


SERVICES  AND  PRODUCTS 


Criteria 


Method  of  Implementation 


The  effectiveness  of  a  given 
type  of  service  or  product 
is  a  function  of  the  prob¬ 
abilities  of  occurrence  of 
all  events  essential  to  that 
service  or  product. 


1. 


The  effectiveness  of  a  given 
service  or  product  is  a 
function  of  the  collective 
effectiveness  values  of  all 
Individual  operations  in 
accomplishing  events  re¬ 
quired  to  produce  it. 


This  can  be  determined  by  measur 
ing,  for  a  population  of  needs, 
the  percentage  of  needs  which 
pass  each  event  required  to  ac¬ 
complish  objectives  of  the 
services  or  to  accomplish  the 
production  of  the  products. 


6.  The  «f fsetiveness  of  a  given 
library  farviee  or  product 
is  relative  to  th«  collective 
indifference  (of  potential 
users,  librarians  and  their 
supervisors )  betveen  that 
service  or  product  and  other 
service#  or  products  needed 
to  meet  the'r  respective 
objectives. 


OPERATIONS 


Criteria 

Method  of  Implementation 

7.  The  efficiency  of  a  given 
library  operation  is  a 
function  of  unit  cost  and 
quality  (effectiveness) 
of  operation  outputs. 

1.  Where  high  correlations  exist 
between  operational  costa  and 
outputs  within  and  among  Army 
Technical  Libraries  :tt  is 
possible  to  develop  attainable 
standard®  of  efficiency  fnt 

1.  This  can  be  determined  by 
subjective  analysis  of  the 
value  or  utility  of  each 
aervice  or  product  in  sup¬ 
porting  the  mission  of  the 
library  and  Che  mission  of  the 
parent  organization. 


given  re-ges  of  effectiveness, 

2.  Efficiency  of  routine  opera¬ 
tions  can  be  measured  against, 
standard  data  when  standard  units 


of  work  are  produced.  Time 
standards  for  routine  opera¬ 
tions  which  are  performed  to 
accomplish  essential  ev-  v  s  in 


library  services  can  be  developed 
for  each  routine  operation 
through  G.A.P.  or  other  work 
earapli.  ’  studies.  Group  ef¬ 
ficiency  can  be  measured  against 


the  standard  e  *  expressed  as 
an  index  of  staff  utilization. 

8.  The  effectiveness  of  • 

1 .  This  can  be  determined  by  the 

given  operation  is  a  func¬ 
tion  of  the  probabilities 
that  essential  events  occur 
due  to  the  outp  s  of  that 
operation. 


relationships  or  correlations 
between  certain  event  probabil 
ities  and  the  outputs  of 
operations . 


<> 


9.  The  ef fectlveness  of  a  given 
operation  Is  a  function  of  its 
output  contribution  to  the  total 
value  of  library  services  or 
products . 


1.  This  can  be  determined  by 
subjective  analysis  of  the 
value  or  utility  of  each 
operations  output  in  adding 
utility  to  various  services 
*  and  products.  _ 


Criteria  for  the  evaluation  of  library  efficiency  and  ef fectlveness , 
methods  of  implementation  and  potential  methods  for  developing  efficiency- 
effectiveness  standards  are  anticipated  as  the  end  product  of  this  study 
in  Phase  III.  It  is  anticipated  that  some  of  the  final  criteria  which 
are  presented  in  Phase  III  will  obviously  be  easily  supported,  some  will 
be  sound,  but  difficult  to  demonstrate  and  still  others  will  be  of  the 
nature  of  hypotheses. 


A  number  of  limits  should  be  considered  at  this  time  in  order  that  the 
contractor's  intent  will  not  be  misinterpreted. 

Limits: 

1.  The  criteria  are  not  intended  to  be  all  inclusive  as  a  basis 
for  management  decisions  concerning  overall  library  efficiency 
or  effectiveness. 

2.  The  methods  of  implementation  of  these  criteria  ere  not,  at 
this  stage  of  the  state-of-the-art  of  library  evaluation, 
proposed  as  being  purely  scientific.  Many  areas  concerning 
efficiency  of  library  operations  and  effectiveness  of  library 
operations  and  services  continue  to  elude  scientific  measure¬ 
ment  or  evaluation.  For  example:  The  findings  of  this  report 
do  not  include  criteria  for  the  measurement  of  (a)  the  value 
of  information  in  a  collection,  (b)  the  value  of  books,  doc¬ 
uments,  etc.,  (c)  the  effectiveness  of  the  library  in  meeting 
individual  client  needs,  (d)  the  efficiency  of  nonroutine 
administrative  operations  such  as  staff  and  users  training. 

However,  the  best  scientific  methods  available  in  the  state- 
of-the-art  of  "scientific”  management  have  been  selected  by 
the  coordinated  judgments  of  3  management  consultants  on  the 
contractor *8  staff.  After  these  methods  have  been  tested  for 
validity  during  Phase  III  of  this  effort,  the  feasibility  of 
their  application  at  various  types  of  libraries  will  be  determined. 
These  validated  criteria  and  methods  of  implementation  are  ex¬ 
pected  to  be  only  one  link  in  the  evolution  from  "artistic  and 
subjective"  library  evaluation  to  "scientific  and  objective" 
library  evaluation. 
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3.  Standards  developed  by  Beans  of  validated  criteria  and  methods 

uf  implementation  should  not  be  regarded  as  being  standards  which 
should  at  once  be  strictly  enforced.  Extensive  testing  and 
further  development  is  necessary  before  standards  of  sufficient 
precision  can  be  developed  to  control  library  efficiency  and 
effectiveness  within  confidence  limits  significantly  tight  to 
Justify  rigid  policies.  However,  the  criteria  and  methods  will 
serve  as  a  basis  for  establishing  adequate  standards  for  perform¬ 
ance  evaluation. 

4.  The  usefulness  of  the  criteria  and  methods  validated  during  this 
study  will  relate,  in  the  f'nal  analysis,  to  the  ability,  available 
time,  and  inclination  of  library  administrators  or  Army  management 
officials  to  adapt  and  implement  them.  Therefore,  it  is  recom¬ 
mended  chat  a  seminar  and/or  a  training  program  be  started  to 
encourage  further  development  and  use  of  the  validated  criteria  for 
measurement  of  efficiency  and  ef icctiveness . 


II.  THE  NATURE  OP  CRITERIA 


It  la  necessary  to  strive  for  some  Intellectual  precision  when  think¬ 
ing  of  the  concepts  connoted  by  the  words  "criterion"  and  "standard."  Further¬ 
more,  there  Is  an  ever-present  danger  of  concept  confusion  when  we  attempt 
to  represent  degrees  of  meeting  standards  in  terms  of  "effectiveness,"  and 
then  quantify  this  somehow  in  terms  of  "efficiency"  when  the  factors  of 
time  and  cost  are  introduced. 

For  these  reasons,  it  is  appropriate  to  discuss  the  nature  of  "criteria." 
From  this,  concepts  of  the  nature  of  "standards"  should  develop,  as  well  as 
ideas  of  how  measures  of  "effectiveness"  emerge  from  degree  of  ability  to 
meet  standards;  "efficiency"  values  Involving  time  and  cost  should  then 
become  obvious. 

"Criterion"  is  defined  in  Webster's  Seventh  New  Collegiate  Dictionary 
as  "a  standard  on  which  a  judgment  or  decision  may  be  based."  The  word 
"standard"  as  a  synonym  for  criterion  is  "something  set  up  and  established 
by  authority  as  a  rule  for  the  measure  of  quantity,  weight,  extent ,  value, 
or  quality."  "Standard,"  "criterion,"  "gauge,"  "yardstick,"  "touchstone" 
denote  a  means  of  determining  what  a  thing  should  be.  "Standard"  applies  to 
any  definite  rule,  principle  or  measure  established  by  authoritv.  "Criterion" 
may  apply  to  anything  used  as  a  test  of  quality  whether  or  not  formulated 
as  a  rule  or  principle. 

It  would  be  advantageous  to  the  progress  of  the  work  of  this  contract 
to  construct  a  model  showing  the  relationships  between  the  concepts  of 
"criterion,"  "standard,"  "effectiveness"  and  "efficiency."  The  model  will 
be  fundamental  to  the  authors'  thinking  and,  although  some  readers  may 
disagree  with  it,  it  will  be  defended  as  the  basis  of  the  system  of  criteria 
eventually  to  be  proposed.  The  model  Is  as  follows: 

CRITERION  -  a  requirement  on  which  a  judgment  or  decision  may  be  based. 

STANDARD  -  something  set  up  or  established  by  authority  ss  a  rule 
for  the  measure  of  quantity,  extent,  value  or  quality.  In  the  sense 
used  here,  a  standard  is  a  criterion  with  qualification,  quantification 
or  dimensions  added. 

EFFECTIVENESS  -  the  extent  or  degree  to  which  a  particular  thing 
fulfills  the  mission,  goal  or  objective  for  which  it  was  performed, 
that  is,  the  degree  to  which  it  meets  the  standard  set  by  authority. 

EFFICIENCY  -  effective  operation  as  measured  by  comparing  pro¬ 
duction  with  cost  (as  in  energy,  time  and  money).  Efficiency 
is  a  measure  of  the  number  of  units  produced  per  unit  of  energy, 
per  unit  of  time,  per  unit  of  money,  etc. 
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Thin  concept  of  "efficiency"  assumes  that  the  units  produced  are 
"effective”  unite,  that  la,  they  meet  the  "standard"  which  authority  has 
established  froa  the  original  "criterion." 

To  draw  a  general  conclusion  froa  the  criterion-standard-effectiveness- 
efficiency  eodel,  it  is  not  possible  to  establish  specific  levels  such  as 
efficiency  without  first  establishing  aore  generic  levels  such  as  effec¬ 
tiveness.  In  turn,  aeaaures  of  "effectiveness"  cannot  be  established 
without  first  establishing  "standards"  and,  more  generic  than  standards, 
criteria. 

Although  analogies  are  not  always  satisfactory  aethods  for  proving  a 
hypothesis,  they  sonatinas  serve  as  helpful  examples.  In  this  sense, 
consider  this  example.  Suppose  criteria  are  to  be  established  for  military 
"commandos."  *By  studying  what  coasumidos  are  expected  to  do,  it  is  noted 
that  they  nust  often  jump  over  fences  to  get  to  the  site  of  their  intended 
operation.  One  criterion  for  determining  the  effectiveness  of  coanandos, 
therefore,  could  be  the  ability  to  juap  fences. 

In  order  to  understand  aore  about  the  importance  of  the  fence-juaplng 
requlrerent ,  it  is  necessary  to  study  this  aspect  of  coaaando  tactics. 
Suppose  a  study  shows  that,  to  reduce  fence-juaplng  to  the  status  of  a 
constant,  it  is  necessary  to  juap  the  highest  fences  found  -  say  a  height 
of  20  feet.  However,  it.  is  known  that  no  aan  unaided  can  juap  20  feet 
high.  Thus,  to  set  a  standard  of  ability  to  juap  20  feet  high  for  coaaando 
candidates  is  unrealistic  and  would  rule  out  all  candidates.  Therefore, 
a  further  study  is  performed  where  the  total  population  of  fences  to  be 
jumped  is  aaapled  and  their  distribution  analyzed.  Suppose  further  that 
this  distribution  curve  shows  that  the  ability  to  juap  6  foot  fences  will 
permit  surswuntlng  95Z  of  all  fences  encountered.  An  "authority"  can  then 
decide  chat  coaaando  candidates  auat  be  able  to  juap  6  foot  fences  and 
this  becomes  the  "standard"  for  that  requireaent. 

Let  it  be  supposed  that  in  the  field  we  aeasure  the  actual  performance 
of  completely  trained,  experienced,  practicing  coanandos  with  reference 
to  jumping  fences.  If  all  coanandos  juap  over  6  foot  fences  with  no  ex¬ 
ception,  their  group  effectiveness  is  100Z  of  the  standard  requireaent, 
and  at  least  95Z  of  the  operational  fence-juaplng  requireaent.  If  some 
of  the  men  jump  fences  higher  than  6  feet,  their  operational  effectlx'eness 
la  higher  than  95Z,  but  lass  than  100Z. 


The  analogy  has  been  chosen  from  an  area  renote  froa  librarlanshlp 
so  that  the  aore  complex  characteristics  of  librarlanshlp  will  not 
coaplicate  the  Issues  being  developed. 
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The  efficiency  of  the  process  can  Chen  be  determined  in  teras  of  the 
time  and  cost  of  jumping  fences  (coses  include  probable  loss  in  casualties, 
energies,  equipment,  etc.). 

The  model  might  be  stated  as: 

CRITERION  -  one  of  the  criteria  for  determining  the  effectiveness:  of  the 
commandos  is  the  ability  to  jump  fences. 

STANDARD  -  in  the  theater  of  commando  operations  where  fence  heights 
up  to  six  feet  include  95Z  of  all  fences  which  are  encountered,  all 
commandos  must  be  able  to  jump  over  fences  up  to  six  feet  high. 

EFFECTIVENESS  MEASURE  -  in  actual  operation  every  commando  jumped 
all  fences  up  to  six  feet  high.  Some  jumped  higher  fences.  None 
jumped  all  fences  more  than  six  feet  high.  As  a  group  the^r  opera¬ 
tional  effectiveness  rating  for  this  particular  criterion  is  at  least 
0.95  but  less  than  1.00.  Effectiveness  with  regard  to  the  standard 
is  1.00. 

EFFICIENCY 


Efficiency  -  a  coat  function  of  jumping  the  fences  in  a  particular 
operational  case. 

Fence-jumping  would,  of  course,  be  only  one  of  a  series  (Cl,  C2,  C3...cn) 
of  commando  criteria.  As  in  the  determination  of  effectiveness  of  electronic 
or  other  equipment, where  overall  reliability  (R)  is  a  function  of  the 
reliability  of  all  essential  component  parts  (assuming  no  shunts) : 

R  •  ri  x  r2  x  r3«..x  rn,  so  the  overall  effectiveness  of  the  commando  would 
be  a  function  of  all  criteria  which  affect  commando  performance: 

E0  -  Ei  x  E2  x  E3...En, 

where: 

Eo  ■  overall  effectiveness  of  commandos  in  accomplishing  the 
goals  and  objectives  of  their  mission; 

Ex  ■  effectiveness  at  jumping  fences; 

E2  •  effectiveness  at  demolition; 

^3,4,5  etc.-  effectiveness  at  other  essential  operational  tasks. 

Depending  upon  the  standards  set,  the  ability  to  meet  the  individual  stands:  s 
determines  the  minimum  effectiveness  values.  The  actual  effectiveness  value 
will  usually  be  higher,  but  not  significantly  so  if  the  standards  are  tight. 

In  relating  this  approach  to  a  library  situation,  the  question  may  be 
asked:  "Why  do  libraries  exist"?  The  answer  could  be  "To  provide  services 
to  clients."  The  effectiveness  of  libraries  would  be  a  measure  of  how  well 
the  libraries  provide  these  services  to  the  clients.  In  order  o  be  more 
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analytical  and  meaningful  in  discussing  the  effectiveness  of  services, 
the  various  kinds  of  services  provided  by  libraries  must  be  identified, 
such  as,  for  example,  (1)  circulation  services,  (2)  reference  search 
services,  (3)  translation  services,  etc.  Even  this  breakdown  does  not 
permit  the  determination  of  effectiveness  at  a  meaningful  level.  Levels 
of  greater  definition  must  be  sought  and  each  individual  item  at  that 
level  must  then  be  studied.  Using  reference  search  services  as  an  example, 
by  analysis  one  can  identify  four  kinds  of  searches,  the  kind  depending 
upon  how  the  client  requests  the  service: 

1.  Unequivocal  expression  of  an  unequivocal  need. 

Example :  A  client  requests  a  copy  of  "Annual  Review  of 
Information  Science  and  Technology,"  Volume  I,  1966, 

American  Documentation  Institute,  Carlos  A.  Cuadra -Editor , 
pub.  by  Interscience  Div.  of  John  Wiley  &  Sons,  New  York. 

II.  Equivocal  expression  of  an  unequivocal  need. 

Example;  The  client  wants  the  particular  book  described 
in  Case  I,  but  does  not  know  the  precise  title  or  the  editor's 
name,  publisher,  etc.,  and  has  to  describe  his  need  ambiguously. 

III.  Equivocal  expression  of  an  equivocal  need. 

Example:  A  client  does  not  know  the  identity  of  specific 
books  or  documents.  Furthermore,  he  does  not  know  precisely 
what  he  is  seeking.  He  may,  for  example,  request  information 
on  the  effect  of  contaminants  on  the  propertlen  of  ruby  lasers. 

IV.  Unequivocal  expression  of  an  equivocal  need. 

Example:  A  client  knows  what  he  is  seeking,  but  does  net  know 
the  publications  which  are  relevant.  He  may,  for  example, 
request  a  list  of  all  books  published  in  Great  Britain  on 
plastics  in  1965. 

Type  III  questions  are  those  which  most  require  expert  subject  index¬ 
ing,*  although  Type  II  and  Type  IV  also  require  them. 

In  studying  the  effectiveness  of  the  library  in  responding  to  Type  III 
search  questions,  what  are  the  factors  involved?  Among  others  are:  (1) 
completeness  of  library  collection  or  "data  bank,"  (2)  subject  Indexing  of 
books,  documents,  etc.,  (3)  skill  and  knowledge  of  reference  librarians. 

*  Cataloging  and  Indexing  are  used  Interchangeably  in  this  text. 
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Assuming  chat  the  library  collection  la  a  constant  and  that  the 
reference  librarian  possesses  sufficient  knowledge  and  skill  to  be  able 
to  locate  all  pertinent  documents  which  are  adequately  Indexed  within  a 
limiting  time,  indexing  will  be  considered  the  only  significant  variable. 
Since  indexing  is  the  major  method  used  to  label  the  things  stored  so 
that  they  can  be  retrieved  upon  demand  by  library  users,  it  is  a  candidate 
criterion  of  the  ability  to  regain  documents  by  search  and  thus  of  the 
effectiveness  in  serving  client  needs. 

To  hypothesize,  let  one  criterion  of  effectiveness  of  search  be  in¬ 
dexing.  Furthermore,  assume  that  the  only  concern  is  with  subject  index 
terms. 

Unlike  the  commando  example,  it  is  difficult  to  determine  real  maximum 
number  of  index  terms.  It  cannot  even  be  established  that  the  maximum 
number  corresponds,  for  a  given  document,  to  the  number  of  different  words 
in  that  document.  Synonymy  could  make  the  maximum  larger.  However,  it  can 
be  assumed  from  experience  that  the  desired  number  of  index  terms  should 
be  in  a  certain  range.  Then  it  is  possible  by  authority  to  specify  a 
number  of  index  terms  and  declare  this  to  be  the  standard.  In  order  for 
an  authority  to  specify  this  number,  it  is  necessary  to  perform  a  study  of, 
say,  the  increasing  probability  of  locating  documents  as  a  function  of  the 
number  of  index  terms  per  document,  as  well  as  a  function  of  the  number  of 
documents  of  that  class  (e.g.,  mathematics,  L.C.  Class  QA;  chemistry,  L.C, 
Class  QD) .  This  probability  is,  of  course,  a  function  of  several  other 
factors  such  as  time  permitted  for  search  (cost)  and  the  skill  of  the 
searcher.  The  first  two  are  significant  factors  and  will  suffice  for  ex¬ 
perimental  development  at  this  point.  The  effect  of  indexing  and  the  number 
of  documents  for  the  Type  III  questions  discussed  earlier  -  where  there  is 
an  equivocal  expression  of  an  equivocal  need  -  may  be  indicated  by  con¬ 
structing  a  three-dimensional  graph  (Figure  1) ,  utilizing  data  from  actual 
searches  to  determine  Pj3  ,  Xjj  and  Yjj  where: 

Pj3  is  the  probability  of  finding  three  or  more  documents  relevant 
to  the  client's  equivocal  expression  of  an  equivocal  need  at  a 
j  library; 

X jj  is  the  total  number  of  documents  in  all  subject  classes  searched, 
e.g.,  mathematics  or  chemistry,  at  library  j  for  query  i  ; 

Yjj  is  average  number  of  index  terms  per  candidate  document  in  the 
relevant  subject  classes  at  library  j  for  query  i  . 
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It  now  becomes  possible  to  gather  data  based  on  document-finding 
results  foi  a  population  of  Typ<*  III  queries,  at  a  number  of  different 
libraries,  where  the  r^an  number  of  index  terms  used  on  documents  re¬ 
called  as  well  aa  the  number  of  relevant  documents  in  the  given  subject 
classes  can  be  determined,  t'.'om  these  data  equations  for  the  probability 
of  finding  some  number  of  documents  can  be  constructed.  For  example,  the 
probability  of  finding  three  or  more  documents  may  s>e  some  function  of 
X|j  jv,and  f.j  ,  1  e.,: 

pj3-‘f  [tXijUYjjUti]  1  ]. 

Here  the  time  factor  "t"  has  been  added  aa  the  third  member  of  the  equation 
and  additional  factors  can  be  Inserted.  The  variable  of  search  ability 
will  be  considered  constant  for  qualified  referei.ee  librarians.  However, 
a  proficiency  test  could  be  administered  to  level  this  factor.  The  sub¬ 
scripts  i  and  j  refer  to  a  particular  query  in  the  i  family  of  queries 
at  a  partlcul  -  library  in  the  j  family  of  libraries.  In  a  group  of  ten 
libraries,  j  “  1,2, 3. ..10  and  an  individual  library,  sar,  has  100  queries 
with  X  j  j  ,  Yjj  and  tj.  known;  these  queries  are  i  »  1,2,3. ..100.  The  first 
query  at  the  first  library  will  have  specific X^  ,  Y  j  j  -'ndtjj  values  of,  say 
X  |;  -  1000  documents,  Y  ||  3  terms  and  f , ;  “  1  hr. 

Having  developed  this  probability  situation  the  following  proposition 
can  be  posed  to  the  library  administrator: 
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If  a  standard  Is  set  which  requires  a  probability  value  P  (perhaps 
952)  that  you  will  be  able  to  retrieve  three  or  more  documents  In  response 
to  client  needs  in  specified  subject  fields,  when  your  collection  size  in 
that  subject  field  is  a  certain  number  of  documents,  and  time  "c"  can  be 
spent  searching,  it  is  necessary  that  these  documents  in  your  collection 
be  indexed  to  an  average  depth  y  .  If  the  document  collection  size  is 
altered,  or  if  "t"  is  altered,  Y  will  change. 

It  will  be  possible  to  prepare  families  of  graphs  for  Pj  values  for 
retrieving  increasing  numbers  of  documents:  1  or  more;  2  or  more;  3  or 
more;  etc. 

Having  determined  the  depth  of  number  of  index  terms  for  a  set  of 
constants,  the  library  authority  can  now  set  a  "standard"  for  indexing 
depth.  While  these  "standard"  conditions  exist,  meeting  them  will  insure 
that  the  library's  effectiveness  in  responding  to  client  search  needs  of 
that  type  has  the  probability  value  chosen  with  a  certain  predetermined 
risk  of  error.  Once  this  number  of  index  terms  is  determined  end  set  by 
authority  as  the  standard  number  of  index  terms  for  that  subject  classifica¬ 
tion  in  that  collection,  adherence  to  that  standard  should  achieve  the 
probability  of  effectiveness  indicated  by  the  study. 

In  this  discussion  it  is  assumed  that  the  index  terms  are  selected 
by  trained  and  experienced  indexers  on  the  basis  of  a  carefully  prepared 
thesaurus . 

It  must  be  emphasized  at  this  point  that  for  the  most  rractical  ap¬ 
plication  of  these  criterla-standards-ef fectivenesa-ef ficieucy  concepts, 
each  situation  should  be  regarded  as  dynamic.  If  indexing,  for  example, 
should  prove  to  be  one  of  the  criteria  of  effectiveness  of  technical 
libraries,  there  may  be  different  standards  for  various  subject  fields 
to  provide  tighter  and  more  reliable  standards,  and  it  is  quite  possible 
that  the  standards  carefully  determined  for  today's  situation  in  a 
particular  subject  field  could  be  quite  different  at  a  later  time.  The 
depth  of  indexing  suitable,  say,  for  archeology,  might  not  equally  serve 
solid-state  physics  or  oxidative  enzymes  and,  within  these  fields,  what 
holds  true  today  might  not  be  true  later.  It  is  likely,  in  an  overall 
dynamic  situation,  that  enforcing  standards  nonetheless  would  maintain 
stable  effectiveness  for  a  short  run  period.  Periodic  measurements  of  the 
effectiveness  probabilities  should  be  taken  to  determine  if  the  enforcement 
of  standards  realizes  the  level  of  required  (or  predicted)  effectiveness. 

At  this  stage  of  the  discussion,  it  should  also  be  brought  out  that 
the  criteria-standards-ef fectiveness-ef f iciency  concepts  are  most  applic¬ 
able  for  overall  library  service  effectiveness.  These  methods  will  measure 
the  effectiveness  of  a  library  in  responding  to  a  population  of  questions' 
service  requests  or  clients.  The  study  of  such  populations  wl'l  give  data 
for  quantifying  that  concept  of  effectiveness. 
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These  considerations  do  not,  however,  give  data  for  quantifying 
the  effectiveness  of  a  library's  response  to  an  individual  client's 
request.  The  latter  is  a  different  kind  of  response  effectiveness  and 
the  distinction  between  the  two  should  be  recognized.  For  clarity,  there¬ 
fore.  let  us  distinguish: 

E#m  *  overall  search  service  effectiveness  in  responding  to 
populations  of  Type  III  search  needs, 

Ejc  •  effectiveness  in  responding  to  the  particular  Type  III 
search  needs  of  individual  clients. 

Our  study  will  provide  data  for  obtaining  values  for  E  jjj.  It  doe# 
not  appear,  however,  that  this  study  will  give  objective  and  quantitative 
values  for  Exc.  To  obtain  values  for  this  measure  of  effectiveness  it  is 
necessary  to  have  a  more  meaningful  knowledge  of  user  feedback,  user 
satisfaction,  user  project  progress,  user  project  importance  or  something 
along  those  lines.  To  develop  this  knowledge  we  must  study  feedback — 
what  la  it,  how  it  can  be  identified  and  how  it  can  be  quantified. 

Some  attempt,  however,  should  be  made  to  identify  certain  types  of 
questions  wh;  h  would  reduce  the  equivocal  nature  of  needs,  for  example, 
questions  which  would  establish  (1)  the  time  deadline  for  receipt  of  in¬ 
formation,  (2)  the  estimated  number  of  relevant  documents  published, 
and  (3)  the  percentage  of  relevant  documents  which  should  satisfy  the  need. 

The  remaining  'uivocal  areas  of  needs  may  be  measured  by  subjective 
and  qualitative  criteria  or  empirical  criteria  in  the  judgment  of  an 
expert  who  would  attempt  to  level  the  effectiveness  of  search  strategy, 
retrieval  strategy,  etc.,  after  exposure  to  the  library  operations  per¬ 
formed  and  user  feedback. 


Personal  communication  with  Cyril  Cleverdon. 
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III.  DATA  AND  INFORMATION  COLLECTION'  AND  ORGANIZATION 


As  the  work  cf  this  contract  developed,  it  became  apparent  that 
there  are  several  approaches  to  developing  criteria  for  evaluating  the 
efficiency  and  effectiveness  of  library  operations  and  services.  The 
various  approaches  lend  themselves  in  different  ways  to  the  numerous 
aspects  of  library  operations  and  the  prods  ~ts  or  services  arising  out 
of  these  operations.  There  are  undoubtedly  additional  approaches  other 
than  chose  we  utilized,  but  attention  will  be  directed  chiefly  to  5 
approaches . 

These  5  approaches  involved  the  utilization  of  both  data  and  informa¬ 
tion.  It  is  the  purpose  of  this  chanter  to  separate  these  data  and  in- 
f  ®ation  into  the  5  classes.  Fo  -  each  approach  the  authors  refer  to  the 
data  or  information  as  <i  "tvpe"  co -responding  to  the  approach  under  discus¬ 
sion.  Although  the  tvpcs  are  discussed  separate!/,  there  will  be  some 
inevitable  concept  overlaps.  These  will  be  minimized  as  far  as  possible. 

The  word  "data"  is  used  in  thi3  report  in  the  broadest  sense.  Actually, 
in  this  work  the  contractor  is  interested  in  gathering  any  type  of  knowl¬ 
edge  which  might  be  useful  in  recognizing  and  establishing  criteria  for 
evaluating  effectiveness  of  library  operations  and  services.  It  is  not 
our  purpose  to  launch  into  a  lengthy  discussion  of  the  distinction  between 
"data"  and  "information."  Rather,  for  simplicity,  the  word  'data"  is  used 
as  connoting  chief lv  numerical,  mathematical  or  statistical  symbolic 
values,  and  the  word  "information"  as  connoting  chiefly  descriptive  facts 
expressed  in  words , 

The  "data"  or  more  properly  the  "information"  required  for  approach 
No.  1  is  simply  a  3et  of  statements  concerning  existing  criteria  gathered 
either  from  the  literature  identified  in  Phase  1  or  from  statements  made 
by  librarians  visited  during  Phase  II. 

The  data  or  information  for  approach  No.  2  comprise  the  existing 
mission  statements  for  the  libraries  of  the  Army  Technical  Library  complex. 
Shortly  after  the  beginning  of  the  contract,  the  Chief  of  the  Scientific 
and  Technical  Information  Branch  of  Picatinnv  Arsenal,  Department  of  the 
Army,  send  a  request  for  mission  statements  to  88  Army  Technical  Libraries. 
These  were  reviewed  by  that  office,  and  c  total  of  72  mission  statements 
was  forwarded  to  us  for  analysis  and  evaluation. 

The  data  for  approach  No.  3  are  statistical  in  nature.  Obtaining 
them  was  simply  a  matter  of  examining  several  existing  reports  and  studies 
on  Army  Technical  Libraries  and  extracting  the  data.  The  analysis  of 
these  data  is  presented  in  another  separate  section. 

The  gathering  of  "data"  for  approaL  s  and  5  became  one  of  the 

tasks  to  be  accomplished  during  the  library  *.  its  of  Phase  II.  It  was  not 
the  intention  of  the  contract  to  visit  all  Army  Technical  Libraries,  but 
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rather  a  sample  of  the  population  representing  selected  aspects  required 
for  study.  The  Army  Technical  Library  complex  is  made  up  of  approximately 
107  libraries.  These  are  widely  separated  geographical'/  and  differ  much 
in  character,  scope,  collections,  staff  size,  location  in  supervisory 
organizations,  missions,  services  and  operations  and  many  other  charater- 
istlcs.  Some  serve  research  organizations,  both  basic  and  applied.  Some 
are  for  the  use  of  developmental  scientific  and  technical  groups.  Some 
are  for  test  and  evaluation  groups.  The  subject  contents  vary  widely, 
covering  the  majority  of  scientific  and  engineering  disciplines  and  sub- 
fields.  In  order  to  make  an  organized  an-*  fairly  logic.31  approach  to  the 
selection  of  10  libraries  for  visits,  it  was  considered  necessary  to  estab¬ 
lish  a  set  of  requirements  bv  which  to  evaluate  the  population  of  the  72 
libraries.  The  sample  chosen  is  based  upon  ten  criteria  and  is  not  a 
completely  random  statistical  sample.  This  method  was  used  in  order  to 
accomplish  the  objectives  of  this  contract  with  a  minimal  sample  size. 

The  ten  criteria  for  sample  member  selection  are  as  follows: 

1.  The  organization  should  have  responded  to  an  earlier  Department 
of  the  Army  in-house  request  for  a  statement  of  its  mission. 

This  request  had  been  made  to  libraries,  information  analysis 
centers  and  technical  information  centers. 

2.  The  organization  should  be  a  technical  library  within  the  im¬ 
plied  definition  of  ATLIS  Report  No.  1,  September,  1966, 

"TECHNICAL  AND  MEDICAL  RESEARCH  LIBRARIES  AND  INFORMATION 
CENTERS  OF  THE  DEPARTMENT  OF  THE  ARMY,"  and  should  be  included 
in  that  study. 

3.  The  library  should  be  in  the  group  for  which  data  were  presented 
in  the  STINFQ  REPORT  No.  i,  13, S.  ARMY  ON-'ITE  SCIENTIFIC  AND 
TECHNICAL  INFORMATION  SURVEY,  September,  1964. 

4.  The  library  should  have  been  included  in  the  Army  Library 
Career  Program  Survev  (except  Corps  of  Engineer  Libraries) . 

5.  The  libraries  should  include  at  least  one  which  utilizes  auto¬ 
matic  data  processing  equipment  for  its  operations  or  services 
as  per  "STUDY  OF  MECHANIZATION  IN  DOD  LIBRARIES  AND  INFORMATION 
CENTERS,"  Booz-Allen  Applied  Research  Inc.,  September,  1966, 

AD  640,  100. 

6.  The  libraries  should  not  be  limited  to  those  in  the  Washington, 
D.C.  metropolitan  area,  but  Army  Technical  Libraries  in  the 
geographical  area  should  be  utilized  if  they  fulfill  other 
requirements  adequately. 

7.  The  population  of  libraries  should,  if  possible,  represent  the 
various  Army  Commands,  such  as  CDC,  AMC,  CONARC,  Corps  of 
Engineers  and  Surgeon  General's  Office. 
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8.  The  libraries  should  include  not  only  Chose  serving  research 
and  development  groups,  hut  also  test  and  evaluation  groups. 

9.  The  libraries  should  represent  as  wide  a  scope  of  collection 
and  staff  sizes  as  possible. 

10.  The  libraries  should  represent  as  wide  a  variety  ct  scientific 
and  technical  subject  matter  in  their  collections  as  possible. 


All  libraries  chosen  for  visits  were  notified  by  letter  by  the  C-iief, 
Scientific  and  Technical  Information  Branch,  Picatinny  Arsenal,  Department 
of  the  Army.  This  notification  requested  permission  for  the  visits, 
specified  thr*  the  visits  would  be  for  1  to  2  days,  named  an  approximate 
date  and  explained  the  purpose  of  the  visit.  As  the  scheduled  date  for 
each  visit  drew  near,  the  head  librarian  of  each  chosen  library  was 
contacted  to  confirm  the  date  and  was  sent  a  packet  of  background  informa¬ 
tion  to  provide  the  library  staff  with  some  of  the  ideas  of  our  research 
team.  The  packet  contained  a  statement  outlining  the  various  criteria 
which  we  visualized  for  development,  and  a  sample  interview  guide  which 
the  research  team  expected  to  use  for  obtaining  the  data.  The  interview 
guide  was  revised  as  a  result  of  experience  gained  in  the  first  few  visits. 
The  length  of  stays  varied  from  one-half  to  two  days.  The  length  depended 
upon  the  extent  to  which  the  staff  had  prepared  itself  as  a  result  of 
studying  the  literature  packet,  reviewing  the  nature  of  the  data  requested, 
and  reviewing  the  data  available  at  the  library.  In  all  visits,  a  tour 
of  the  library  was  made,  and  conversations  held  with  several  members  of 
the  library  staff  to  obtain  some  idea  of  attitude  toward  management, 
clientele  and  fellow  staff  members.  The  major  part  of  the  visit  time, 
however,  was  utilized  to  obtain  the  requested  data. 


In  general  the  data  sought  were  available  or  easily  derived  from 
records  kept  at  the  Army  Technical  Libraries  or  from  past  studies  on  Army 
Technical  Libraries.  However,  certain  classes  of  data  were  more  difficult 
to  obtain,  such  as  user  statistics  (number  of  actual  and  potential  users, 
extent  of  in-house  use,  types  of  users),  use  and  extent  of  reference 
services,  extent  of  bibliographic  service,  weights  of  library  operations 
as  they  contribute  to  each  service,  and  weights  of  each  service  as  they 
contribute  to  the  library  mission  and  objectives.  In  order  to  get  these 
data,  ranges  or  estimates  were  accepted.  The  mean  was  estima  ed  fre  the 
range  and  estimates  of  other  parameters  such  as  the  mode  and  Median  Using 
these  parameters  the  mean  was  estimated  and  was  given  confidence  limits 
which  were  within  +  152,  902  of  the  time  for  the  data  re  ordeJ.  All  other 
data  sought  had  in  general  been  derived  from  the  statistics  kept  in  each 
library  or  from  statistics  obtained  from  CE1R  studies,  p  evio  is  ATL1S 
studies,  library  fact  sheets  and  other  previous  studies.  The  accuracy 
of  these  data  cannot  be  estimated  by  comparison,  since  the  data  represent 
changing  situations  over  a  period  of  years.  However,  due  to  the  acces¬ 
sibility  of  these  data,  they  are  expected  tc  be  accurate  within  +  152 
902  of  the  time  or  better. 

Further  investigation  on  the  reliability  of  data  is  r presented  in 
the  statistical  analysis  in  the  following  chapters. 
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IV.  TYPE  1  APPROACH  EVALUATION  AND  ANALYSIS 
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The  Idee  behind  this  approach  Is  the  rather  obvious  one  of  simply 
determining  what  criteria  already  exist,  either  in  Army  Technical 
Libraries,  other  techt. 'cal  libraries  or  other  libraries.  This  approach 
was  to  Identify  these  "criteria"  either  from  the  literature,  (see  Phase  I 
abstracts),  or  from  questions  asked  during  visits  to  the  selected  Army 
Technical  Libraries.  Not  all  criteria  may  be  plainly  labeled  "criteria"; 
they  may  also  go  under  such  other  labels  as  "standards"  or  "policies"  or 
"instructions"  or  a  variety  of  other  terms. 

Since  it  is  not  the  purpose  of  the  present  contract  necessarily  to 
invent  criteria,  but  rather  to  identify  or  recognize  as  well  as  to  orig¬ 
inate  such  concepts,  simply  to  locate  such  as  already  exist  would  serve 
the  purpose  equally  well.  If  suitable  criteria  already  exist,  it  will 
serve  the  task  adequately,  therefore,  if  the  contractor  simply  identifies 
them  and  validates  them.  Criteria  are  not  par  sc  independent  and  adequate 
in  a  practical  sense  for  determining  effectiveness  of  libraries  or,  for 
that  matter,  c*  any  other  organization.  It  is  necessary  t  at  they  be 
supported  by  additional  and  more  specific  concepts  such  a  ire  found  In 
"standards."  Effectiveness"  can  then  be  determined  by  measuring  the 
degree  to  which  the  item  under  consideration  meets  the  standard.  If  we 
wieh  to  attach  time  and  cost  parameters,  "efficiency"  can  be  established. 
Furthermore,  criteria  should  not  be  vague  or  ambiguous,  but  cles^cut  and 
capable  of  development  into  easily  understood  and  enforceable  standards. 

It  was  the  experience  of  the  contractor,  in  searching  the  literature, 
that  statements  of  criteria  ot-en  did  not  have  any  organized  concept  in 
mind  and  were  difficult  to  apply  In  a  practical  wav.  However,  there  are 
many  useful  standards  for  which  there  are  no  criteria  statements.  In 
tome  cases,  it  is  not  necessary  to  state  these  criteria  explicitly, 
because  they  are  Implicit  to  any  on*  knowledgeable  and  experienced  in  the 
field.  In  some  cases,  however,  It  would  be  helpful  to  every  one  involved, 
especially  nonlibrarian  administrators  with  library  administration 
responsibilities,  if  the  coordination  of  the  criteria,  the  standards, 
and  the  measurement  of  effectiveness  were  made  more  clearly  visible  and 
understandable , 

The  following  discussion  of  existing  criteria  is  not  exhaustive. 

Since  it  is  fairly  obvious  that  not  all  criteria  are  plainly  labeled  as 
such,  and  that  some  criteria  mav  be  enunciated  in  reports  or  other  pub¬ 
lications  not  included  in  the  literature  search,  it  is  quite  possible  that 
existing  important  ones  mav  have  been  missed.  Furthermore,  it  is  quite 
possible  that  some  important  criteria  are  so  much  taken  for  granted  as  to 
have  been  overlooked.  It  is  hoped,  however,  that  the  most  important  ones 
will  be  taken  into  consideration. 
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The  criteria  gathered  from  the  literature  appear  to  fall  Into  four 
general  categories: 

(1)  library  mission  and  organization-related  criteria; 

(2)  administrative,  management  and  general  criteria; 

(3)  operations  criteria; 

(4)  service  criteria. 

The  criteria  associated  with  the  mission  of  the  library  usually  are 
goal-related.  For  example,  some  existing  criteria  state:  (1)  the  library 
should  support  the  total  organizational  program  and  goal  (378)*;  (2)  the 
library  goals  should  support  the  goals  of  the  parent  organization  (455)  ; 

(3)  the  library  goals  must  be  explicit  as  to  extent  of  service  and  priority 
requirements  (455) ;  (4)  the  role  of  the  library  is  determined  largely  by 
requirements  for  library  services  of  the  laboratory  served  (207).  In  one 
case  a  criterion  was  subject-related:  a  scientific  library's  mission  is 
to  provide  scientific  information  (248) . 

Some  of  the  criteria  statements  for  management  personnel  discuss 
inter-organizational  expectations  and  some  discuss  more  functionally  what 
an  administrator  should  do.  This  can  be  advantageous,  when  the  administrator 
of  the  library  is  fully  aware  of  the  criteria.  These  statements  are:  (1)  a 
college  library  should  be  directly  responsible  to  the  college  president 
(182) ;  (2)  a  library  should  publicize  its  services  so  that  the  users  know 
they  exist  (345),  (24);  and  (3)  library  goals  must  be  agreed  upon  by 
management,  library  and  users  (455). 

Very  few  criteria  exist  for  a  library  budget.  Both  of  the  criteria 
found  were  in  the  1964  SLA  Objectives  and  Standards  for  Special  Llbrarles(770) . 
These  were:  (1)  a  budget  for  a  special  library  should  be  based  on  recom¬ 
mendations  of  the  library  administrator;  (2)  the  library  administrator  has 
the  responsibility  and  authority  for  the  expenditure  of  his  budgeted  funds. 

The  criteria  for  determining  staff  personnel  were  more  difficult  to 
locate  precisely.  These  dealt  mostly  with  professional  requirements. 

These  were  for  example:  (1)  The  criterion  of  a  profession  is  the  existence 
of  a  systematic  body  of  knowledge  of  substantial  intellectual  content.  A 
member  of  the  profession  should  have  personal  skill  in  the  application  of 
this  knowledge  to  specific  cases  (109).  (2)  An  adequate  professional 

library  staff  is  an  essential  component  of  basic  research  facilities  (176)  . 

(3)  The  library  staff  should  include  persons  trained  in  the  users’  fields 
and  be  familiar  with  their  problems  (225).  (4)  The  librarians  should 

have  knowledge  of  sources  of  information  and  library  training  (381).  (5) 

The  library  staff  is  responsible  for  the  efficient  organization  of  materials 
and  for  making  available  the  catalogs  and  indexes  for  prompt  access  to  the 
materials  and  the  information  by  the  patrons  (770)  . 


Numbers  in  parentheses  refer  tc  references  in  Phase  I  report  c  the 
supplemental  references  in  this  report. 
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Other  criteria  are  more  specific,  and  might  even  be  considered 
standards,  as  far  as  staffing  Is  concerned.  These  are:  (1)  A  Job 
description  should  Include  a  statement  In  detail  of  actual  activities. 
Indicating  Importance  of  each  activity,  the  conditions  under  which  the 
job  Is  performed,  and  the  materials  needed  to  carry  out  the  job  (428); 

(2)  In  order  to  determine  grade  level  In  a  government  library,  the 
knowledge  requirements,  scope  of  assignment  and  level  of  responsibility 
should  be  factors  (94);  and  (3)  To  qualify  as  a  librarian,  a  person 
should  have  five  years  of  formal  education  beyond  secondary  school  in¬ 
cluding  graduation  from  library  school  (11). 

Many  more  criteria  statements  exist  for  the  operations  or  technical 
processes  which  are  carried  out  In  the  library.  In  fact,  the  criteria 
for  the  acquisitions  program  of  a  library  determines  the  nature,  size 
and  complexity  of  the  collection.  A  criterion  for  cataloging  is  the 
comparative  ease  with  which  items  can  be  located. 

Some  or  the  criteria  statements  pertinent  to  acquisitions  are: 

(1)  A  special  library  should  acquire  materials  and  Information  for  the 
organization's  current  and  future  needs  (770)  ;  (2)  The  selection  criteria 
should  Inform  the  library  staff  as  to  the  Identity  of  the  material  to 
be  processed  and  also  Indicate  to  the  clients  what  they  can  expect  from 
the  library  (424);  (3)  The  organization's  information  requirements  should 

provide  an  accurate  selection  of  all  forms  of  information  to  fulfill  the 
needs  of  research  projects  and  the  scientific  and  technical  interests  of 
group  members  (258);  (4)  Subject  coverage  of  the  collection  should  be 
intensive  and  extensive  enough  to  meet  the  current  and  anticipated  In¬ 
formation  requirements  of  the  library  clientele  (770)  ;  (5)  The  size  of  the 
collection  should  depend  upon  the  amount  of  material  pertinent  to  the 
organization's  needs  (770)  ;  (6)  The  library  should  have  as  much  of  the 
needed  material  on  hand  as  feasible  to  anticipate  needs  (206)  ;  and  (7)  The 
collection  can  be  compared  with  published  lists  of  key  literature  (261). 

Much  has  been  written  about  the  use  of  relevance  and  recall  ratios 
a3  measures  of  effectiveness.  There  Is  considerable  controversy  as  to 
the  validity  of  recall  and  relevance  ratios  as  being  measures  of  indexing 
systems  effectiveness.  Although  criteria  of  this  nature  were  not  con¬ 
sidered  at  this  time  for  cataloging  and  classification, their  validity  will 
be  discussed,  in  Phase  III  of  this  contract.  Others  found  to  pertain  to 
cataloging  and  classification  were:  (1)  The  catalog  should  be  in  enough 
detail  for  the  users  to  be  able  to  utilize  it  efficiently  (211);  (2)  The 
catalog  and  classification  should  be  done  in  such  a  manner  as  to  assure 
the  highest  possible  accuracy  and  consistency  for  the  user  (218);  (3) 

Some  criteria  for  a  qualitative  catalog  listed  by  Oliver  Lllley  (244)  were: 
cost;  physical  size;  time  to  construct,  maintain,  search;  scope  by  topic; 
ease  in  determining  relevance;  nuuber  of  access  points  as  main  entry; 
universality  of  information;  rate  of  growth  per  new  entry;  obsolescence 
rate;  simplicity  of  apparatus;  adaptability  to  reproduction  and  dissemina¬ 
tion;  adequacy  in  supplementing  similar  tools. 
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There  do  not  seem  to  be  as  many  criteria  existing  for  library 
services.  Criteria  for  circulation  services  include:  (1)  Ease  in  ac¬ 
cessibility  is  the  most  desired  characteristic  of  a  system  (236) ;  (2) 

The  physical  location  of  the  materials  should  be  determined  by  the  amount 
of  use  (143);  (3)  Success  in  locating  library  material  is  more  signif¬ 
icant  than  the  number  of  volumes  in  the  collection  (456) ;  (4)  The  ease 
of  access  or  obtaining  the  materials  is  the  most  desired  characteristic 
of  a  system  (456,  236);  and  (5)  The  use  of  the  collection  by  students  in 
a  college  library  is  the  ultimate  test  of  the  collection  and  the  library 
(292). 

The  fewest  criteria  exist  for  the  reference,  search  and  bibliographic 
types  of  services.  Some  examples  are:  (1)  Technical  information  service 
should  be  provided  as  a  background  for  specific  or  individual  projects 
(207);  (2)  The  special  library  should  provide  successful  reference  service 
varying  from  answering  miscellaneous  questions  to  providing  literature 
surveys  and  comprehensive  bibliographic  reports  (235);  and  (3)  The  library 
staff  should  locate  library  materials  and  information  promptly  upon  re¬ 
quest  (770)  - 

The  library  visits  by  the  research  team  were  not  fruitful  sources  of 
criteria  statements  or  concepts.  However,  the  subjects  discussed  and  the 
data  gathered  during  the  visits  have  a  direct  or  Indirect  relation  to  the 
effectiveness  or  the  efficiency  of  the  library  in  performing  its  operations 
or  providing  its  services  or  products.  Therefore,  it  is  judged  necessary 
to  Include  in  this  report  some  of  the  observations  made  during  the  visits 
over  and  above  the  actual  numerical  data.  These  observations  pertain  for 
the  most  part  to  library  methodology  or  procedural  matters  rather  than  to 
the  more  generic  concepts  usually  thought  of  as  constituting  effectiveness 
criteria.  Nonetheless,  some  of  these  procedural  matters  might  constitute 
focal  points  from  which  to  develop  criteria. 

Some  interesting  points  arise  with  reference  to  management  matters 
which  have  considerable  bearing  on  both  effectiveness  and  efficiency.  One 
of  these  has  to  do  with  accountability  records.  It  appears  for  the  most 
part  that  accountability  records  are  kept  only  for  acquisitions  which 
must  be  purchased.  This  is,  of  course,  understandable  because  it  is  neces¬ 
sary  to  account  for  direct  expenditure  of  funds.  There  is  a  tendency, 
however,  to  view  "free"  acquisitions  as  not  costing  anything.  But  there 
are  the  costs  of  ordering,  following-up,  receiving,  processing,  storing 
and  retrieving  these  "free"  documents  which  most  often  represent  a  larger 
investment  in  documents  than  the  purchase  price. 

A  second  point  of  Interest  noted  on  the  visits  which  would  have 
direct  bearing  on  library  efficiency,  although  it  would  not  be  Immediately 
obvious  because  the  costs  are  hidden,  is  the  necessity  to  respond  to  Army- 
wide  or  higher  echelon  regulations  which  may  have  little  or  no  bearing  on 
library  operation.  That  is  to  say,  in  a  puristic  sense,  there  are  certain 
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regulations  which  technical  libraries  must  abide  dv  for  purposes  of  higher 
management,  jhich  do  not  wake  a  positive  contribution  to  library  effective¬ 
ness  but  do  have  it  negative  influence  on  librarv  efficiency  by  requiring 
staff  time.  Certain  procurement  r  quirements  are  examples.  For  instance, 
accountability  requirements  consume  labor  man-hours  which  may  be  utilized 
•ore  effectively  on  other  taaka.  Utilitv  Analysis,  a  method  discussed  in 
chapter  VIII,  is  a  tool  #jr  determining  the  moat  effective  and  efficient 
allocation  of  man-hours  to  operations.  After  Utility  Analysis  of  the 
value  of  accountability  op*  ttiona  as  op, osed  to  the  value  of  other  opera¬ 
tions,  it  may  be  discovered  that  ccouncabilitv  requirements  have  a  negative 
Influence  upon  library  efficiency  and  poss^  .lv  upon  library  effectiveness. 

The  following  ooservat Iona  summarize  some  of  the  findings  with  respect 
to  library  methods  and  procedures. 

Activities  which  are  a  parr,  of  acquisition*  and  accessions: 

Library  contents  selection  -  Not  all  libraries  consider  selection  of 
nev  books  by  the  user  staff  to  be  a  part  of  acquisitions.  In  moat  libraries, 
tha  librarian  aelact*  the  material  with  suggestions  from  the  users  and 
library  staff.  About  half  of  the  libraries  can  order  any  books  and  doc- 
usanta  that  they  wiah  without  further  approval,  as  long  as  the  materials 
acquired  are  within  the  scope  and  budget  of  the  ’lbrarv.  In  most  cases, 
hovevar,  there  was  no  firm  policy.  The  degree  of  librarian  professionalism 
determined  where  selection  of  new  materials  for  the  librarv  was  initiated. 

Checking  -  Moat  libraries  check  to  see  if  they  have  the  publication 
requastad  before  placing  the  erdsr  for  its  acquisition. 

Ordering  -  The  biggest  ielav  in  acquisitions  is  in  ordering.  Depending 
upon  the  installation,  or  th*  overall  command  of  the  library,  there  are 
varying  amounts  of  paper  work  Involved. 

Receipt  -  Good  receipt  records  arc  kept  at  all  places,  in  sanv  cases 
in  the  form  of  the  shelf  list  which  is  s  matter  of  ccisplving  with  Amv 
regulations  on  accountability . 

Documents  -  Alaost  all  documents  are  received  free.  No  accountability 
records  are  required  f  r  free  documents. 

Computerization  -  ibnly  one  librarv  visited  was  using  a  computer  for 
providing  a  record  of  items  on  order  and  received. 

Activities  _p  er  formed  as__p  art  o  f  cl  a  *  s  i  f  v  ing  ai .  d  c  ataloging 

Classification  -  The  libraries,  all  technical  ir.  nature,  do  not  .all 
uee  Librarv  oT  Congress  Classification  numbers.  Some  use  the  Dewev  Decistal 
Classification  System.  No  other  classification  systems  were  found  to  he 
used  for  books. 


Cataloging  -  Cataloging  was  generally  interpreted  to  mean  using 
authority  listings  to  check  correct  author  entries,  using  ALA  rules  for 
subject  headings,  or  Library  of  Congress  subject  headings.  About  one- 
half  of  the  libraries  did  their  own  cataloging  and  classifying,  and  about 
one-half  bought  their  cards  from  the  Library  of  Congress.  Even  those 
which  purchased  cards  from  the  Library  of  Congress  either  altered  the 
cards  or  had  to  catalog  and  classify  some^ books  themselves.  Collection 
size  was  not  the  determinant  of  whether  the  cataloging  and  classification 
was  done  in-house  or  depended  on  Library  of  Congress  cards.  However,  the 
size  of  the  staff  did  reflect  whether  or  not  in-house  cataloging  was  done 
since  additional  staff  were  needed  to  perform  this  operation.  One  library 
was  in  the  midst  of  converting  from  Dewey  Decimal  Classification  to  Library 
of  Congress  Classification. 

Verifying  -  Checking  to  determine  if  books  are  classified  the  same 
wav  every  time  was  performed  in  all  libraries,  using  either  authority 
files  or  the  National  Union  Catalog. 

Shelf  listing  -  This  function  was  performed  in  all  libraries  and  pro¬ 
vided  the  data  for  material  accountability. 

Catalog  maintenance  -  All  libraries  attempted  to  keep  their  catalog 
current  in  order  to  reflect  weeding  and  new  acquisitions. 

Document  Indexing  -  A  variety  of  documents  is  acquired  by  the  various 
libraries.  Of  these  documents,  a  large  percentage  originates  in  federal 
government  agencies  and  may  be  acquired  via  government  announcement  bulletins. 
Even  though  such  bulletins  provide  indexing  entries,  most  of  the  libraries 
visited  do  their  own  indexing  of  the  documents. 

It  is  not  common  for  the  libraries  to  index  journal  articles,  even 
though  they  may  be  pertinent  to  the  particular  library's  collection.  The 
one  library  visited  which  does  index  journal  articles  pertinent  to  its  col¬ 
lection  consequently  spends  a  greater  proportion  of  time  on  indexing  than 
do  the  other  libraries. 

Preparation  and  maintenance  of  library  materials: 

This  was  considered  to  include  book  marking,  book  pocketing,  weeding 
and  card  catalog  maintenance.  This  library  operation,  although  not  obvious 
to  the  library  user,  takes  about  5%  to  10Z  of  the  library's  time,  accord¬ 
ing  to  the  data  gathered  from  these  libraries.  Most  of  the  policies  or 
standards  in  this  area  are  written,  but  are  out  of  date;  therefore,  saying 
that  most  of  these  standards  are  tradition  would  be  more  correct. 

Circulation : 

Circulation  was  considered  in  two  ways:  one,  as  pertaining  to  material 
circulating  from  a  circulation  desk,  upon  request  by  a  user;  the  other, 
a  predetermined  circulation  or  routing  of  journals,  reports  and  documents. 
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The  total  amount  of  time  spent  on  both  tvpes  of  circulation  varied  from 
about  10Z  to  30Z  of  staff  time.  Since  the  actual  activities  for  these 
two  types  of  circulation  vary,  questions  were  asked  separately  about  each. 

Circulation  upon  request  -  Some  libraries  required  the  patron  to  be 
registered.  This  simplified  having  to  check  lists  when  the  user  wanted 
some  classified  materials,  so  that  he  did  not  have  to  wait.  Most  libraries 
permitted  browsing  in  the  stacks.  All  libraries  required  that  the  doc¬ 
uments  and  books  be  signed  out.  Although  a  staff  member  was  located  near 
the  entrance  of  the  library  to  take  care  of  these  circulation  activities, 
this  person's  time  was  not  totally  taken  up  by  signing  books  out.  In 
some  cases  thev  aided  in  returning  the  books  to  the  shelves,  received  the 
mail,  answered  some  reference  questions  and  scanned  journals  for  articles 
of  interest  to  other  personnel.  The  number  of  unit9  circulated  per  day 
related  quite  closely  to  the  amount  of  time  spent  on  this  activity.  All 
libraries  had  active  circulation  of  their  materials. 

The  policies  and  standards  for  circulation  of  items  on  request  are 
both  written  and  traditional.  Those  libraries  with  significant  amounts 
of  such  circulation  have  written  policies. 

Charging  -  All  books  and  documents  should  be  signed  out.  In  sore 
cases,  books  are  charged  out  without  having  been  cataloged  if  there  Is 
an  immediate  need  for  them. 

Overdues  -  The  libraries  generally  were  not  very  strict  on  overriues. 
Most  loan  periods  are  30  davs.  When  loan  periods  are  exceeded,  notices 
are  sent  out.  There  was  no  wav  of  determining  if  this  loan  period  i: 
adequate,  because  renewal  statistics  are  not  kept. 

Reserves  -  .or  the  most  part,  the  Armv  Technical  Libraries  did  not 
provide  reserves.  However,  thev  do  have  collections  which  do  not  circulate. 

Interlibrarv  loans  -All  libraries  practiced  interlibrarv  loans. 
Depending  upon  the  librarv's  location  and  the  subject  field,  some  libraries 
were  more  involved  in  interlibrarv  loans  than  others. 

Photocopying  -  The  patrons  are  generally  free  to  have  anv  material 
copied  as  long  as  thev  adhere  to  the  new  copyright  laws.  Almost  all  of 
the  libraries  have  photocopying  equipment  themselves,  or  equipment  nearby 
for  their  use. 

Routing  of  Journals  or  predetermined  circulation  -  All  but  one  of 
the  libraries  provide  extensive  circulation  of  magazines.  This  library 
service  was  found  to  be  one  most  usuallv  automated,  if  the  library  is 
using  anv  automatic  or  electronic  data  processing  equipment. 
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Reference : 

Reference  questions  can  be  either  short,  asking  for  an  answer  or  a 
fact  or  even  an  article  or  pertinent  item,  or  thev  ran  he  more  extensive 
asking  for  several  references,  a  bibliography,  or  an  ana  I  vs  is  of  informa¬ 
tion  contained  in  the  published  literature.  In  some  rare  instances,  the 
short  reference  questions  can  take  longer  to  answer  than  is  required  for 
the  comnilatlon  of  a  short  bibliography.  It  was  observed  that  the  short 
reference  questions  are  not  asked  to  be  defined  hv  the  patron  nearly  as 
often  as  are  requests  for  a  bibliographic  tvpe  of  reference  search.  The 
compilation  of  a  bibliography  almost  alwavs  implies  extensive  work.  How¬ 
ever,  if  the  short  reference  questions  were  defined  better,  probablv  less 
time  could  be  spent  responding  to  them.  Reference  is  one  of  the  few 
library  services  which  brings  the  staff  in  direct  contact  with  both  the 
user  and  with  library  materials,  books,  catalogs  and  other  library  tools. 
Even  so,  it  appears  as  though  this  activity  takes  no  more  than  30 7.  of 
the  staff  time;  the  usual  amount  of  time  is  about  20%. 

Bibliographic  services  varv  greatly  from  library  to  library  because 
of  differences  in  staff  capabilities,  extent  of  the  collections  and  staff 
sizes.  In  some  cases  detailed  bibliographic  work  is  done  within  the  in¬ 
formation  center  or  by  subject  specialists  closely  related  to  the  library 
activities.  Most  libraries  which  provide  bibliographies  take  advantage 
of  the  bibliographic  services  of  the  Defense  Documentation  Center,  the 
National  Library  of  Medicine  and  the  National  Aeronautics  and  Space 
Administration,  Scientific  and  Technical  Aerospace  Report  Facility. 
Libraries  do  not  show  partiality  as  to  how  requests  for  information  come 
in,  as  by  phone,  by  mail  or  during  personal  visits.  The  preference  is 
determined  bv  the  person  making  the  request. 

Only  two  of  the  libraries  have  equipment  on  which  thev  can  perform 
reference  searches  automatically  or  index  their  collections.  Because  the 
most  frequent  bibliographic  searching  areas  are  government  reports  or 
documents,  the  Defense  Documentation  Center  has  automated  this  service 
and  made  it  available.  The  libraries  have  made  use  of  this  service. 

No  set  policies  appear  to  exist  for  preparing  bibliographies.  When 
they  are  requested,  they  are  prepared,  if  the  capability  exists  within 
the  library. 

Translation: 

Only  one  library  prepares  translations.  The  libraries  usually  accept 
the  requests  for  translations  and  attempt  to  have  them  supplied,  either 
by  arranging  for  the  translation  or  ascertaining  that  they  already  exist 
elsewhere.  All  translations  requested  are  usually  supplied  since  their 
acquisition  has  already  been  okayed. 
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Library  Publications: 


Library  publications  which  are  in  the  form  of  published  accessions 
lists  are  prepared  in  everv  library.  Some  lists  contain  abstracts  or 
annotations  of  selected  articles.  This  form  of  publication  taK.es  little 
time  to  prepare,  since  it  is  made  up  from  catalog  cards.  In  some  esses 
the  document  accessions  are  also  listed.  These  iists  are  usually  arranged 
by  subject,  but  in  some  cases  by  the  issuing  agenev.  Document  lists  ace 
not  frequent,  since  most  of  the  documents  are  classified. 

These  lists  are  circulated  to  the  library's  natrons  and,  to  some 
miner  extent,  outside  the  grout  of  immediate  library  users.  The  publica¬ 
tions  are  thought  to  be  well  received,  since  there  are  always  requests  for 
the  items  listed. 

Public  Rela t tons : 

Public  relations  are  considered  as  those  activities  which  inform 
the  users  about  the  library  either  bv  orientation  tours,  preparation  of 
brochures,  reports  to  management  or  preparing  various  liorarv  exhibits. 

All  libraries  perform  these  activities;  they  took  less  than  5X  of  the 
total  time. 

Budget : 

The  libraries  were  found  to  have  no  set  budget,  for  the  moat  part. 

The  chief  librarian  and  the  division  chief,  If  the  library  is  j.n  an 
information  center,  usually  prepared  the  budget.  There  was  no  set  policy 
or  standard  evident  for  determining  the  budget,  but  the  amount  of  funds 
available  to  the  library  depends  upon  the  amount  of  monev  authorized  for 
the  installation  or  command. 

Other  administration  activities : 

Preparation  of  library  standards  -  These  are  performed  by  the  head 
librarian  or  by  management  personnel  outside  the  library.  In  order  to 
measure  efficiency  or  effectiveness  of  the  librarv,  several  libraries 
use  work  measurement  time- standards  developed  by  personnel  outside  the 
library. 

Library  committee  role  -  About  half  of  the  libraries  ave  library 
committees.  The  committees  appear  to  have  limited  power  in  most  cases, 
but  provide  functional  review  and  advice  to  the  libraries.  In  one  case, 
the  library  committee  runs  the  library. 

Staffing  policies  -  In  some  cases  the  head  librarian  is  able  to 
specify  to  management  the  job  to  be  done  and  the  personnel  needed  to  do 
the  Job.  The  usual  policy  la  for  the  comptroller's  office  or  the  manpower 
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office  to  determine  the  nir-ber  of  positions  and  the  types  of  persons 
needed  to  fill  them.  Other  libraries  use  patron  reactions  or  immediate 
supervisor  review  to  rate  the  library  and  staff  needs. 

Library  space  and  size  -  In  general  space  made  available  to  the 
library  is  effectively  used.  This  is  usually  determined  by  man&r,’ment 
outside  the  library. 

Other  library  functions: 

Of  the  miscellaneous  remaining  functions  and  tasks  which  might  be 
performed  in  a  technical  library ,  the  librarians  queried  replied  that  they 
do  not  perform  any  other  operation  or  produce  any  other  service  which  re¬ 
quires  more  than  a  total  of  5%  of  their  time.  However,  after  examining  the 
amounts  of  time  allotted  to  each  activity  by  administrators  it  was  noted 
that  total  time  added  up  to  only  about  75%.  It  appears  that  the  admin¬ 
istrators  did  not  always  include  themselves  in  the  accounts  of  time  spent 
on  various  service-producing  activities  and  also  did  not  appear  to  realize 
the  true  magnitude  of  time  required  for  administrative  duties.  The  time 
of  the  other  library  staff  employees  was  well  accounted  for. 


V.  TYPE  2  APPROACH  EVALUATION  AND  ANALYSIS 


This  effort  was  aimed  a£  gathering  the  mission  statements  of  the 
Array  Technical  Libraries  as  they  exist  at  the  present  time  in  official 
records.  It  was  then  the  purpose  to  examine  these  to  determine  if  out 
of  this  analysis  common  denominators  would  emerge  which  would  constitute 
the  basis  for  criteria  statements 

Mission  statements  were  requested  of  Armv  Technical  Libraries  by 
the  Chief  of  the  Scientific  and  Technical  Information  Branch  of  Picatinny 
Arsenal,  Department  of  the  Armv.  This  was  done  early  in  the  contract 
period  to  permit  sufficient  time  to  receive  the  maximum  number  of  state¬ 
ments  for  the  analysis.  The  libraries  were  instructed  to  send  the 
statements  as  they  existed  at  that  time,  and  not  to  prepare  statements 
specifically  in  response  to  the  request.  The  requests  were  couched  in 
terms  asking  for  migsion  statements  and  organization  charts  of  the  or¬ 
ganizational  group  administratively  responsible  for  the  library's 
operations  and  services,  and  of  the  installation  where  the  library  is 
located.  Out  of  the  total  population  of  approximately  120  libraries 
identified  as  Army  Technical  Libraries,  requests  were  sene  ro  88  and  72 
responses  were  received.  Three  of  there  were  eliminated  because  they 
were  atypical;  the  analysis  was  performed  on  the  remaining  69  statements. 

Although  this  discussion  of  Type  2  data  is  closelv  related  to  that 
of  the  Type  5  data,  we  wish  to  make  a  distinction.  In  this  section  (Type  2) 
we  are  discussing  the  analysis  of  data  or  information  representing  library 
situations  as  thev  existed  at  the  time  of  the  statement  requests,  Tn  the 
Type  5  situation  we  discuss  conditions  which  the  librarian  should  recognize 
and  recommendations  related  to  these  conditions. 

It  appears  fairly  common  to  establish  for  anv  organizational  unit 
certain  statements  which  give  it  form  and  purpose.  They  are  often  presented 
in  a  more  or  less  hierarchical  arrangement,  expressing  first  the  generic 
concepts  and  progressing  to  more  and  more  specific  statements.  These 
statements  exnr^sa  the  policies  to  be  followed  bv  the  organization  and 
often  the  exact  functions,  procedures  or  operations  of  the  unit.  One  such 
model  of  statements,  which  we  judge  provides  the  necessary  framework  for 
an  organization,  can  be  shown  as  follows: 

MISSION  -  usually  a  broad  and  somewhat  philosophical  statement  which  ex¬ 
presses  the  reason  for  existence  of  the  organizational  unit.  It  is  the 
unit's  raison  d'etre. 

COALS  AND  OBJECTIVES  -  these  statements  tend  to  give  fora  and  substance 
to  the  mission  statements.  Thev  classify  major  functions  the  unit  is 
charged  to  accomplish . 

ACTIVITIES  -  these  statements  define  the  actions  which  will  convert 
the  more  generic  goals  and  objectives,  often  rather  static  concepts 
at  best,  into  dynamic  and  meaningful  outputs. 
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OPERATIONS  -  these  statements  specify  the  fairlv  repetitive, 
although  often  hlghlv  p-of essional  processes  which  mu  :  be 
performed  to  produce  the  activity  or  service  output. 

It  is  theoretically  possible  to  develop  such  a  ’’package1'  of  state¬ 
ments  for  any  organizational  unit,  regardless  of  which  echelon  it  oc¬ 
cupies  in  an  organizational  management  universe.  In  an  industrial  world, 
it  can  be  applied  to  the  highest  corporate  ieveis  to  establish  corporation 
policies  and  to  implement  these  in  various  subcompany  products  or  services. 
In  a  government  situation  it  can  be  applied  to  the  executive  branch,  to 
departments,  subdepartments ,  commands,  and  so  on.  More  often  than  not, 
the  more  specific  statements  of  higher  level  organizational  units  express 
the  more  generic  aspects  of  the  organizational  units  beneath  them. 

In  the  overall  task  of  collecting  library  mission  statements  and 
analyzing  them  to  determine  their  characteristics ,  their  ommon  denomina¬ 
tors  as  well  as  their  points  of  difference,  their  orientations  and  their 
obviously  divergent  interpretations  of  chat  the  mission  composers  thought 
libraries  are  supposed  to  do,  a  fundamental  rationale  predominates.  This 
basic  reasoning  is  to  tne  point  that  "ef fecti veness"  of  library  services 
is  undoubtedly  some  function  of  how  well  the  library  accomplishes  the 
purpose  for  which  it  was  established.  This  in  turn  will  depend  upon  how 
well  the  purposes  are  spelled  out  and  explained  and  how  attainable  they 
are  in  reality. 

This  does  not  mean  that  the  missions,  goals  and  objectives,  activities 
or  services,  and  operations  of  all  libraries,  or  of  all  technical  libraries, 
or  even  of  all  Army  Technical  Libraries,  will  be  identical.  But  it  does 
suggest  that,  for  a  given  subpopulation  of  libraries  such  as  Armv  Technical 
Libraries,  there  will  be  a  common  essence  to  mission  statements  as  well  as 
to  the  aubstatementa  under  missions.  The  variations  in  the  statements 
should  be  a  function  of  the  mission  of  the  parent  organization.  The 
missions  anc  their  subordinate  statements  could  constitute  a  basis  for 
judging  effectiveness. 

The  impression  should  not  be  gained  t'.iat  the  contract  investigators 
expected  to  find  complete  uniformity  throughout  the  Armv  lechnical  Library 
system  with  regard  to  the  location  of  the  library  in  the  organization  it 
serves  or  with  regard  to  the  mode  of  expression  of  its  mission  and  sub¬ 
ordinate  services  and  cperations.  It  is  recognized  that  there  is  much 
variation  in  size,  scope,  purpose,  funding  and  other  characteristics  of 
Army  Technical  Libraries,  due  to  their  role  in  supporting  the  mission 
of  their  respective  parent  organizations.  The  purpose  of  the  analysis  is 
to  seek  identification  of  elements  from  which  to  distill  principles  for 
the  establishment  of  the  most  desirable  conditions  bv  which  to  provide 
criteria  for  measuring  effectiveness  independent  of  the  library  size,  scope, 
funding  and  purpose. 
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A  few  words  on  what  are  considered  to  be  a  ewe  Important  aspects  of 
Amy  Technical  Libraries  will  provide  a  little  background  on  the  author's 
concepts  of  these  organizations.  These  can  then  be  areas  of  agreement  or 
controversy  in  ensuing  discussions.  Armv  Technical  Libraries  are  "support" 
organizations.  They  are  -  ot  oddities,  existing  simply  for  the  sake  of 
existence.  They  are.  meant  to  support  the  Army  organizations  or  commands 
of  which  they  are  a  part.  The  libraries  are  intrarelated  with  other 
activities  of  their  own  command  and  interrelated  with  other  libraries  in 
other  Army  commands.  They  are  a  part  of  an  informal  network  and  have 
similar  reasons  for  existence. 

Considering  the  Army  c>  :«imand  levels  alone,  each  command  ha®  its  own 
reason  for  existence,  and  each  is  different  from  its  fellow  commands.  Thus, 
within  Army  commands.  Technical  Libraries  should  indicate  obvious  points 
of  simila^itv,  whereas  the  Technical  Libraries  of  different  Army  commands 
might  be  expected  to  differ  somewhat  because  of  difference  in  command 
missions.  For  these  reasons  we  might  expect  that  an  analysis  of  69  Army 
Technical  Library  missions  should  reveal  that  the  organizations  are  in 
many  respects  quite  similar  and  In  manv  respects  uniquely  different. 

In  order  to  perform  an  analvsis  of  the  missions  statements  received 
from  these  various  Array  Technical  Libraries,  and  to  examine  the  rela  »n- 
ships  of  these  libraries  to  the  Army  organizational  groups  with  which 
they  are  associated,  it  was  decided  that  one  approach  would  be  to  adopt 
what  might  be  termed  a  "standard"  model,  depicting  how  mission  statements 
and  their  subordinate  clauses  might  be  presented  and  to  determine  how  well 
the  actual  mission  statements  received  "fit"  the  model.  The  model  selected 
is  that  presented  on  pages  30  and  31. 

By  way  of  explanation  of  what  might  be  expected  for  the  statements 
of  mission,  goals  and  objectives,  activities  o  services  and  operations, 
we  assigned  beforehand  such  examples  as: 

MISSION  -  Serve  as  a  principal  source  of  published  and  controlled 

information  in  support  of  the  research  and  development  prograss _ _ _ 

(none  of  organization)  and  of  the  separate  commands  located  at  the 
installation . 

GOALS  AND  OBJECTIVES 

(1)  Collect  all  applicable  published  information. 

(2)  Maintain  it  under  bibliographic  control, 

(3)  Disseminate  it,  or  products  resulting  from  processing  it. 

(4)  Conduct  studies  to  determine  needs  of  clients  and 
develop  plans  for  meeting  the  needs, 

(5)  Conduct  research  in  new  or  advanced  methods  of  librarianship 
and  information  handling.  Develop  plans  for  introduction  of 
such  methods  which  would  be  more  efficient  or  cost-effective. 

(6)  Maintain  liaison  with  related  library  or  information  organizations. 

(7)  Others. 
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ACTIVITIES  (SERVICES) 


(1)  Provide  circulation  services. 

(2)  Provide  subject  matter  search  services. 

(3)  Prepare  bibliogia 

(4)  Provide  "current  awareness"  service. 

(5)  Provide  question-answering  service. 

(6)  Provide  translation  service. 

(7)  Provide  abstracting  service. 

(8)  Provide  reference  services. 

(9)  Provide  interlibrary  loan  service. 

(10)  Others. 

OPERATIONS 

(1)  Acquisition  of  books,  documents,  etc. 

(2)  Catalog  books,  documents,  etc. 

(3)  Prepare  materials  for  shelving. 

(4)  Perform  weeding. 

(5)  Store  and  retrieve  books,  documents,  etc. 

(6)  Maintain  shelf  list. 

(7)  Maintain  property  accountability  records. 

(8)  Maintain  records  of  journal  subscriptions,  holdings,  etc. 

(9)  Maintain  binding  operations. 

(10)  Others. 

As  we  study  this  model,  it  becomes  quite  obvious  that  the  responsibility 
and,  for  that  matter,  the  capability  for  establishment  of  statements  at  the 
different  levela  of  specificity  shift  considerably.  For  example,  at  the 
generic  level  the  responsibility  for  preparation  and  establishment  of  the 
mission  statement  quite  possibly  belongs  one  or  more  level  above  the  library 
Itself.  Thus,  if  the  library  is  situated  in  the  Research  and  Development 
Directorate  of  an  organization,  the  library  mission  statement  responsibility 
quite  properly  belongs  to  the  head  of  this  directorate.  The  goals  and 
objectives  statements  can  quite  properly  and  adequately  be  prepared  by  the 
directorate  with  the  cooperation  of  the  head  librarian.  The  activities 
statements,  however,  are  the  direct  responsibility  of  the  head  librarian. 

The  operations  statements  are  functions  of  librarianship  and  the  responsibility 
of  the  librarian  or  lover  echelon  professional  personnel  on  his  staff. 

We  may  say  then,  as  shown  on  page  32,  that  responsibility  for  prepara¬ 
tion  of  the  various  levels  of  statements  should  be  consonant  with  levels  of 
policy-making,  and  with  levels  of  library  "know-how." 
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These  Betters  appear  to  be  rsther  obviou3  and  cheir  importance  m^iv 
tend  to  be  overlooked.  They  play  an  important  part  in  the  overall  concept 
of  library  effectiveness  and  efficiency,  however,  in  that  misplacement 
of  responsibility  for  establishment  and  carrying  out  what  these  statements 
connote  can  result  in  striving  for  misunderstood  ends  and  thus  a  low  oraer 
of  effectiveness.  It  aav  not  be  at  all  sensible  for  the  head  librarian 
to  establish  his  own  library's  uission,  for  he  aav  not  have  a  complete  or 
sufficiently  deep  understanding  of  the  higher  echelon  purposes.  Yet,  it 
is  not  sensible  for  the  higher  echelon  authorities  to  tell  the  librarian 
what  operations  he  should  carry  out  to  accomplish  the  production  of 
certain  services,  for  in  doing  so  they  are  weakening  his  ability  to  carrv 
out  fullv  the  expert  ’ ibtarianship  capabilities  for  which  he  was  engaged. 

This  aspect  of  library  effectiveness  is  also  tied  in  intimately  to 
the  placement  of  the  Techrical  Library  in  its  parent  organization  -  It  is 
•ssentisl  that  Che  library  be  located  such  that  the  next  higher  echelon 
of  authority  haa  the  capability  of  appreciating  and  understanding  com¬ 
pletely  the  technical  aspects  of  the  library's  work  and  Is  not  in  such 
a  position  that  he  has  conflicts  of  interest  between  the  groups  served  bv 
the  library.  Otherwise,  there  should  be  a  willingness  to  delegate  tech¬ 
nical  authority  either  impartially  to  the  technical  groups  the  library 
serves,  or  even  to  the  library  itself.  If  certain  groups,  indl-.iduals 
or  projects  are  to  have  priority,  the  mission  statement  should  reflect 
the  degree  of  priority.  Technical  understanding  and  cooperation  goes  far 
to  lubricate  the  discussion  bv  which  to  determine  kinds  and  degree#  of 
services  to  be  provided  to  each  group  served,  directions  to  take,  and 
budgets  required.  All  these  considerations  are  criteria  for  evaluating 
the  effectiveness  of  the  technical  library. 
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An  examination  of  the  mission  statements  returned  by  the  libraries, 
with  reference  to  location  or  placement  of  the  library  within  the  parent 
organization  it  serves,  gave  evidence  of  considerable  variation.  The  re¬ 
sponses  indicated,  as  would  be  expected,  that  the  installations  differ 
considerably.  The  mission  statements  of  the  parent  organizations  reflected 
the  differences,  and  thus  the  supporting  elements  for  the  installations,  as 
for  example,  the  libraries,  showed  considerable  difference  in  missions. 
Perhaps  as  a  consequence  of  this  the  level  within  which  the  libraries  are 
situated  varied.  Although  this  was  most  noticeable  between  commands,  it 
was  evident  even  within  the  same  command.  The  technical  libraries  associated 
with  the  Army  schools  of  CONARC  exhibited  uniformity  of  placement,  being 
administered  by  the  school  secretary.  This  office  ic  usually  charged  with 
administrative  services,  personal  services,  security  and  the  library. 

In  the  Array  Materiel  Command  there  appears  to  be  no  set  pattern.  Even 
in  those  cases  where  the  effort  is  concerned  with  basic  and  applied  research 
on  assigned  materiel,  the  libraries  do  not  appear  to  be  uniformly  placed. 

The  most  common  locations  were  in  the  Technical  Services  Directorate,  the 
Administrative  Services  and  the  Research  and  Development  Command.  The 
Administrative  Services  Division  or  office  is  usually  concerned  with  mail, 
records,  records  management,  reproduction  and  publishing.  In  very  few  in¬ 
stances  are  the  technical  libraries  directly  under  the  administration  of  the 
reeearch  laboratory  organization. 

Most  of  the  technical  libraries  in  the  Surgeon  General's  Office  are 
hospital  libraries,  serving  the  general  staff  as  well  as  hospital  research. 
These  libraries  are  most  often  placed  in  the  Administrative  Services  Division 
of  the  organization.  Several  of  the  Surgeon  General  libraries  not  at 
hospitals  are  those  lr.  the  Research  and  Development  Command.  These  are 
located  in  the  Support  Division. 

The  technical  libraries  of  the  Corps  of  Engineers  are  commonly  found 
in  the  Office  of  Administrative  Services.  If,  however,  the  library  is  in 
a  research  facility,  it  appears  to  be  directed  by  the  research  center  or 
facility  management. 

The  Combat  Development  Command,  like  others  studied,  appeared  most 
commonly  to  assign  its  technical  libraries  to  the  Administrative  Services 
Division.  The  examination  of  tne  other  commands  revealed  that  there  appears 
to  be  no  set  pattern  as  to  where  the  technical  libraries  are  located  in 
relation  to  the  groups  they  serve  most  closely. 

The  mission  statement  packages  returned  by  the  69  libraries  were  ex¬ 
amined  to  determine  if  the  statements  contain  the  elements  set  >t  in  the 
"model"  previously  devised  for  comparison.  Of  the  packages  ret  ned,  13 
Army  Technical  Libraries  provided  statements  containing  all  4  e'  ments  of 
the  "model."  Thirty-nine  libraries  set  forth  "mission"  statemer.  s.  It 
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appeared  chat  these  statements  for  the  library  were  provided  not  only  in 
the  library  statements,  but  occasionally  in  the  statement  of  the  organiza¬ 
tional  group  administratively  responsible  for  the  librar-. 

The  statements  most  commonly  found  for  the  libraries  might  be  termed 
more  appropriately  "activity"  or  "service"  statements,  rather  than  "mission" 
statements.  It  may  be  that  this  is  explainable  by  the  fact  that  the  mission 
statement  exists  elsewhere  and  thae  the  libraries  are  concerned  solely  with 
the  services  they  are  charged  with  providing.  There  may,  of  course,  be  some 
misinterpretation  of  what  a  library  mission  statement  should  be.  Forty-five 
of  the  libraries  provided  activities  statements.  Of  these,  11  were  mentioned 
outside  of  the  library  statements.  Thirty-four  of  the  libraries  showed 
"gods  and  objectives"  type  statements.  Of  these,  7  appeared  outside  the 
library  statements.  The  "operations"  the  library  is  expected  to  perform 
were  found  in  statements  for  33  libraries.  In  6  cases  these  statements  of 
library  operations  were  found  outside  the  library  statements.  This  location 
of  "operations"  statements  is  unusual,  because  it  would  appear  that  the 
library  staff  would  be  most  aware  of  the  best  operational  techniques  for 
accomplishing  activities  or  services  and  that  these  statements  would  ap¬ 
propriately  appear  in  statements  by  the  library. 

From  the  data  provided,  it  is  difficult  to  determine  if  there  is  any 
difference  from  command  to  command  in  insuring  that  the  technical  libraries 
concerned  have  mission  statements,  and  if  the  mission  statements  and  their 
corollaries  provide  all  elements  necessary  to  give  adequate  and  clear 
guidance  for  library  management. 


VI.  TYPE  3  APPROACH  EVALUATION  AND  ANALYSIS 


The  approach  visualized  here  Is  to  examine  data  which  may  show  possible 
relationships  between  library  statistics  and  efficiency  and  effectiveness 
either  (1)  already  available  in  government-sponsored  contract  or  in-house 
reports,*  and  (2)  data  which  may  be  collected  to  respond  to  questions  in 
greater  depth. 

In  the  case  of  (1),  one  of  the  purposes  of  the  library  visits  by  the 
research  team  was  to  validate  some  of  the  data  in  such  reports.  For  example, 
some  of  the  questions  asked  in  one  of  these  studies  included  queries  on 
whether  or  not  the  Army  Technical  Libraries  provide  services  such  as  loens, 
selective  dissemination  of  information,  shelving/filing/reshelving ,  circula¬ 
tion,  inventory  control,  abstracting,  translations,  etc.  The  validation  of 
such  "yes  or  no"  data  could  make  it  possible  to  develop  criteria  for  Army 
Technical  Libraries  pertinent  to  selected  services  or  operations. 

With  respect  to  (2),  the  research  team  wished  to  collect  additional 
numerical  data,  cost  data,  "yes  or  no"  data,  or  other  qualitative  or 
quantitative  data.  For  example,  depending  upon  the  development  of  the  sub¬ 
ject,  it  becomes  desirable  to  seek  information  or  data  on  such  items  as  the 
following: 


(A)  Number  of  items  in  the  library  collection 

(1)  Books 

(2)  Journals 

(3)  Reports 

(4)  etc. 

(B)  Number  of  items  in 

(1)  Chemistry 

(2)  Physics 

(3)  Biology 

(4)  etc. 

(C)  Clientele 

(1)  Chemist 

(2)  Physicist 

(3)  Biologist 

(4)  etc. 

(5)  PhD 

(6)  MS 

(7)  BS 

(8)  etc. 

*  REPORT  NUMBER  STINFO  1,  U.S.  ARMY,  ON-SITE  SCIENTIFIC  AND  T  CHNICAL 
INFORMATION  SURVEY,  September,  1964,  Contract  No.  D/  49-09 ;  4RO-23, 
CEIR,  or  ATLIS  Report  No.  1,  TECHNICAL  AND  MEDICAL  RESEARCH  LIBRARIES 
AND  INFORMATION  CENTERS  OF  THE  DEPARTMENT  OF  THE  AR.MY,  Sept;  Tiber,  1966. 
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(u)  Programs  or  projects  of  parent  organization 

(1)  Chemistry 

(2)  Physics 

(3)  Biologv 

( 4 )  Materials 

(3)  Missiles 

(6)  etc. 

From  these  data  a  sufficiently  detailed  picture  of  library  missions, 
staff  size,  collection  size,  number  of  users,  services  use,  operations 
outputs,  operatic.. .3  costs  and  other  parameters  could  be  obtained  from  which 
to  draw  some  conclusions  about  relationships  of  efficiency  and  effectiveness 
in  developing  criteria  for  Array  Technical  Libraries. 

The  surveys  and  reports  examined  include: 

1.  "FEDERAL  DEPARTMENTAL  LIBRARIES:  A  SUMMARY  REPORT  OF  A  SURVEY 
AND  A  CONFERENCE,"  Luther  H.  Evans  et  ai.,  Washington:  Brookings  Institute, 
1963.  The  date  foi  this  survey,  the  first  federal  library  survey  made 
since  1376,  were  initially  collected  in  1959-196G  and  included  all  federal 
libraries,  both  general  and  special.  The  survey  was  performed  with  a  three- 
part  cue.Uionnaite.  Part  A  was  filled  out  by  the  administrative  officer  of 
'.he  department  or  agency  or  by  a  representative  designated  by  him.  It  dealt 
with  overall  policy  and  administrative  problems  involved  in  the  operation 
of  the  library  or  Iibra^iEs  bv  the  agencies.  The  questions  included  the 
general  subjects:  I.  sicn  of  the  agency;  II.  Agencies'  libraries;  III. 
Functions  performed  bv  .gency  headquarters  for  the  field  libraries;  IV, 

Budget  of  the  agency;  V.  Future  development. 

Part  B  was  filled  out  by  anv  library  designated  as  reference  and  tech¬ 
nical.  The  questions  for  this  part  were  presented  under  the  following  head¬ 
ings:  I.  Purpose,  scope  and  coverage  of  the  library;  II.  Organization  of  trie 
lioiary,  ill.  Size  and  nature  of  present  collection;  IV.  Acquisitions  policies 
and  procedures;  V.  Binding;  VI.  Cataloging  and  classification;  VII,  Reference 
and  bibliographic  service;  VIII.  Lending  service;  IX,  Library  space;  X. 
Budget;  XI.  Personnel;  XII,  Reporting  of  statistics;  XIII.  Cooperative  ar¬ 
rangement  with  other  libraries;  XIV.  Service  on  Committee  and  relations 
with  assoc  *t ions;  XV,  Future  development;  XVI.  Mechanical  and  electronic 
devices;  XVII.  Miscellaneous.  Part  C  was  intended  for  general  libraries 
and  the  questions  fell  in  the  same  areas  as  Part  B. 

Of  the  libraries  designated  as  Army  Technical  Libraries,  about  half 
had  been  included  in  the  Brookings  survey.  For  the  purposes  of  the  present 
John  I.  Thompson  &  Company  contract,  the  Brookings  survey  data  were  com¬ 
pared,  therefore,  with  present  mission  statements  for  libraries  included 
in  library  visits  in  our  research  to  determine  if  any  major  changes  had 
taken  place  ot  if  their  missions  had  changed  since  1939.  In  the  intervening 


time,  hcvever,  the  DoD  has  undergone  several  reorganisations.  Some 
libraries  are  no  longer  in  the  same  command  or  have  the  same  administration. 
However,  the  primary  subject  missions  appear  to  have  stayed  the  same.  This 
was  determined  by  examining  the  data  supplied  concerning  the  major  subject 
areas  of  the  collection  and  the  tvpe  of  usage. 

2.  "U.S.  ARf'Y  ON-SITE  SCIENTIFIC  AND  TECHNICAL  INFORMATION  SURVEY” 

Report  Number  STINFO  1,  performed  bv  CEIR,  Inc.,  Arlington,  Virginia,  for 
the  Director  of  Army  Technical  Information,  Department  of  the  Army  Contract 
DA  49-Q92-AKO-23,  Final  Report  September,  1964,  AD  445-800.  The  purpose 
of  this  study  was  to  identify  by  means  of  an  on-site  survey  the  personnel, 
frnilities  and  funding  (for  FY  1°64)  involved  in  the  oerformance  of  scientific 
and  technical  information  fur.'-  Ions.  The  objective  was  to  determine  what 
was  being  done  in  the  areas  of  originating,  processing,  storing  and  dis¬ 
seminating  scientific  snd  technical  information,  Pro-" easing  includes  all 
functions,  processes  and  activities  to  which  this  information  is  subjected, 
except  for  the  origination  of  rough  draft  and  the  use  made  of  the  inforaa- 
•-lon  after  the  ultimate  user  has  received  it.  In  addition  to  inventorying 
the  activities  to  which  the  Army  subjects  its  scientific  and  technical  in¬ 
formation,  the  survey  aimed  at  locating  significant  depositories  or  collec¬ 
tions  of  items  containing  data  or  inf c  rmatior.  about  numerous  scientific  and 
technical  subjects  of  interest  to  the  Army. 

The  objective  of  our  research  team  in  analyzing  the  data  of  the  CEIR 
import  was  to  determine  what  trends  could  be  detected  in  certain  selected 
functions.  It  was  judged  that  the  establishment  of  trends  by  a  statistically 
supported  population  of  libraries  might  in  Itself  provide  a  bauis  for  con¬ 
sideration  as  criteria.  The  activities,  services,  functions  and  other 
librar,  or  inforraation-center-related  items  included  in  the  CEIR  survey 
are  reflected  in  the  following  listing. 

ITEMS  IN  CEIR  SURVEY 


Code 

Activity  Name 

Group 

01 

Managing  aTINFO  activities 

STINFO 

Administration 

02* 

Education  of  SliNFO  users 

STINFO 

Administration 

03* 

Coordinating  STINFO  activities 

STINFO 

Administration 

04* 

RAD  in  Information  Sciences 

STINFO 

Administration 

05* 

Planned  technical  visits 

STINFO 

Administration 

06 

Analyzing  results  of  meetings 

STINFO 

Administration 

07* 

Selection  of  items 

Acquisition  of  STINFO 

Items 

08* 

Dollar  procurement 

Acquisition  of  STINFO 

Items 
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09*  Exchange  agreement 

10*  Loans 

11*  Non-dollsr  procurement 

12  Technical  writing 

13  Technical  editing 

14  Technical  review 

15  Preparation  of  format 

16  Printing 

17  "Dummy"  master  preparation 

18  Folding/collating 

19  Binding/stitching 

20  Still  photography 

21  Motion  picture  photography 

22  II lust rating /char ting 

23  Drafting 

24  Model/aculpting/exhibit-maktnft 

25  Flipping 

26  Sound  recording 

27  Photographic  copying 

28  Mi crophotography 

29  Xerography 

30*  Disseminating  new  STINFO 

31*  Disseminating  info  u:  STINFO 

32*  Selective  dissemination 

33*  Use  of  displays 

34*  Secondary  distribution 


Acquisition  of  STINFO  Items 
Acquisition  of  STINFO  Item3 
Acquisition  of  STINFO  Items 
Primary  Report  Preparation 
Primary  Report  Preparation 
Primary  Report  Preparation 
Primary  Report  Preparation 
Report  Production 
Report  Production 
Report  Production 
Report  Production 
Visual  and  Audio  Arts 
Visual  and  Audi;  .Arts 
ViU:.\u  v.nd  Audio  Arts 
Visual  and  Audio  Arts 
Visual  and  Audio  Arts 
Visual  and  Audio  Arts 
Visual  and  Audio  Arts 
Reproduction 
Reproduction 
Reproduct i on 
Dissemination 
Dissemination 
Dissemination 
Dissemination 
Dissemination 
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35* 

Prep,  for  storing/handlint 

Storing  Handling  of  STINFO 

Items 

36* 

She! ving/f iling/reshelving 

Storing  Handling  of  STINFO 

Items 

37* 

Circulation 

Storing  Handling  of  STINFO 

I  terns 

38* 

Inventory  control 

Storing  Handling  of  STINFO 

Items 

39* 

Weeding 

Storing  Handling  of  STINFO 

Items 

40* 

Descriptive  cataloging 

Cataloging 

41* 

Subject  classif ication/ indexing 

Cataloging 

42* 

Indicative  abstracting 

Abstracting  Translating 

4  3* 

Informative  abstracting 

Abstracting  Translating 

44* 

Translation 

Abstracting  Translating 

45* 

"Ready  reference" 

Reference 

46* 

Noncritical  searches 

Reference 

47* 

Critical  searches 

Reference 

48* 

Noneritlcal  summaries 

Reference 

49* 

State-of-the-art 

State-of-t'ne  Art  Review 

50 

Card/tape  punching 

Punch  Cara  and  EDPM  Processing 

51 

EDPM  Systems  design 

Punch  Card  and  EDPM  Processing 

52 

EDPM  programming 

Punch  Card  and  EDPM  Processing 

53 

Plugboard  layout/wiring 

Punch  Card  and  EDPM  Processing 

54 

EAM/EDPM  operation 

Punch  Card  and  EDPM  Processing 

Of  these  items,  29  were  selected  as  appearing  to  be  common  to  technical 
libraries.  These  are:  02,  03,  04,  05,  07,  08,  09,  10,  11,  30,  31,  32,  33, 
34,  35,  36,  37,  38,  39,  40,  41,  42,  43,  44,  45,  46,  47,  48  and  49  (see 
asterisks  ). 
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Sows  difficulty  was  encountered  in  extracting  the  desired  data*  The 
survey  often  dealt  with  entire  information  centers  or  other  information 
organisations  and  not  only  with  libraries.  It  was,  therefore,  necessary 
to  determine  which  of  the  chosen  activities  were  being  performed  in  the 
library  when  the  desired  library  was  part  of  an  information  center.  (One 
observation  made  with  reference  to  the  survey  was  that,  although  it  very 
directly  concerned  libraries,  library  personnel  often  had  only  a  minor  part 
in  supplying  the  data.)  In  order  to  obtain  the  desired  information  regard¬ 
ing  the  selected  functions,  it  was  necessary  to  examine  carefully  the  data 
presented  in  CEIR  Table  1  (Organizational  Elements  by  Code) ,  Table  2 
(Organization  by  Installation  Directory)  and  Table  7  (Elements  within 
Organizations  performing  Activities).  These  tables  are  presented  in  Ap¬ 
pendix  A  of  the  Final  CEIR  Report. 

Of  the  69  Army  Technical  Libraries  which  replied  to  the  requests  for 
information  concerning  their  missions,  organization  and  staffing,  42  had 
also  been  a  part  of  the  CEIR  study.  Our  investigators,  therefore,  ex¬ 
tracted  from  the  survey  information  on  the  29  ons  listed  by  their 

CEIR  code  numbers,  as  they  had  been  replied  to  by  the  42  libraries.  The 
resulting  data  are  displayed  in  the  bar  chart  in  Figure  2.  The  height 
of  the  bars  depicts  the  number  of  libraries  in  the  total  population  of 
42  which  perform  the  activity  or  function  listed. 

3.  "STUDY  OF  MECHANIZATION  IN  DoP  LIBRARIES  AND  INFORMATION  CENTERS.” 
Boox-Allen  Applied  Research,  Inc  ,  Contract  DSA-7-15489,  performed  for 
Defense  Supply  Agency,  Department  of  the  Army,  Sethesda,  Maryland,  1966. 
The  report  summarizes  an  cn-site  survey  of  mechanization  as  it  appeared  to 
be  in  1966.  Presentations  are  included  on  thesaurus  building,  file  struc¬ 
ture,  input  processing,  serial  control,  selective  dissemination  of  in- 
fenaation,  circulation  control,  equipment,  recommendations  on  information 
retrieval  systems,  observations  on  organization,  operation  and  application, 
and  summaries  on  mechanization  status,  ^cope  and  size  of  collections,  and 
CQSATI  subject  aren  breakdowns.  The  survey  covered  76  facilities  and 
individual  reports  were  prepared  for  35.  Among  other  uses,  this  survey 
could  be  used  to  show  some  relationships  between  the  mission  of  t  organ¬ 
izations,  the  number  of  persons  required  to  carry  out  the  services  and 
operations  performed,  and  whether  an  important  service  or  operation  had 
been  mechanized  to  provide  more  efficient  service  and  reduce  the  required 
number  of  manhours. 

Although  the  Booz-Allen  study  contained  an  insufficient  number  of 
the  Army  Technical  Libraries  to  make  the  information  adequately  comparable, 
■any  interesting  data  on  collections  of  books,  periodicals  and  reports, 
classified  according  to  particular  subject  categories,  were  presented. 

4.  "TECHNICAL  AND  MEDICAL  RESEARCH  LIBRARIES  AND  INFORMATION  CENTERS 
OF  THE  DEPARTMENT  OF  THE  *RMY,"  Army  Technical  Library  Improvement  Studies 
(ATLIS)  Report  No.  1,  Washington,  1966.  collected  information  on  the  size 
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and  makeup  of  Artsy  Technical  Library  staffs,  as  well  as  the  composition 
and  size  of  the  collections  at  each  Installation,  As  will  be  discussed 
later,  the  data  from  this  survey  were  examined  in  our  study  to  note  any 
changes  in  size  from  the  more  recently  collected  data  as  well  as  that 
collected  during  the  10  library  visits, 

5.  "ARMY  CIVILIAN  CAREER  PROCRAM  FOR  LIBRARIANS.  LIBRARY  FACT  SHEET," 
Office  of  the  Adjutant  General,  Department  of  the  Army,  Washington,  1966, 
provides  the  following  information  regarding  Army  libraries:  number  and 
type  of  resources,  number  and  type  of  full  time  personnel,  and  the  library 
services  made  available.  Because  the  libraries  of  the  Army  Corps  of 
Fngineers  were  not  included,  it  was  not.  possible  to  make  comparisons  of 
this  important  facet.  As  will  be  discussed  later,  the  data  included  in 
this  fact  sheet  were  employed  in  s'me  of  the  correlation  studies  as  part 

of  the  present  study. 

6.  "SURVEY  OF  SPECIAL  LIBRARIES  SERVING  THE  FEDERAL  GOVERNMENT,  1965," 
Frank  Schick,  Federal  Library  Committee,  Washington,  1967,  Contract  No,  OEG- 
4-7-001913-1913,  1967,  presents  detailed  data  on  the  special  libraries 
serving  the  Federal  Government.  Included  are  data  concerning  organiza¬ 
tional  patterns,  library  clientele  and  activities,  resources  of  library 
materials  and  staff  and  expenditures.  Data  are  also  given  for  the  primary 
subject  areas  of  the  collections  and  the  primary  occupations  of  their 
users.  Library  functions  performed  traditionally  and  by  automated  means 
were  tabulated.  The  information  3  night  in  this  survey  were  quite  similar 

to  that  of  our  library  visit  interview  guide.  For  the  purposes  of  the 
correlations  made  in  our  study,  the  Schick  data  were  included. 

Tables  1  through  6  summarize  some  of  the  data  extracted  from  several 
of  these  referenced  surveys,  as  well  as  some  of  the  data  collected  by  our 
investigators  on  the  10  library  visits.  Although  the  ident'ties  of  the 
10  libraries  surveyed  In  all  5  studies  are  not  revealed,  the  data  are,  of 
course,  directly  comparable.  Data  are  shown  for  staff,  analyzed  according 
to  professional,  library  assistants,  clerical  and  total.  Data  are  also 
shown  for  collections,  classified  according  to  whether  or  not  the  items 
are  books,  volumes,  pamphlets,  reports  and  so  on,  according  to  tf  *  word 
usage  in  the  various  survey  reports.  Because  it  was  fudged  that  the  weed¬ 
ing  policies  of  the  various  libraries  would  have  an  influence  on  th  d  ta 
presented  on  collection  size.  Table  7  is  included,  tabulating  whether  or 
not  a  weeding  step  is  performed.  Table  8  notes  the  observations  taken 
from  the  CF.IR  report  regarding  whether  or  not  the  various  coded  items  are 
included  in  the  same  population  cf  the  10  libraries,  as  well  a.  observa¬ 
tions  made  by  our  investigators. 

Before  discussing  the  treatment  of  some  of  the  data  used  for  study¬ 
ing  correlations,  !t  is  thought  appropriate  to  make  a  few  observations 
concerning  the  data  in  general.  When  we  examine  the  data  from  the  past 
studies,  as  well  as  those  gathered  during  the  present  study,  the  lack  of 
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I  BR  ARY  NO.  BROOKINGS  ATI  IS  I  FLG  FACTSHEET  VISIT 

1 V59  1964  1  965  1  966  1  967 


Table  2  -  Volumes  (Excluding  bound  serlols) 


LIB IRA1Y  NO. 

BROOKINGS 

1959 

AT  US  1 
1964 

FLC 

1965 

FACTSHEET 

1966 

VISIT 

1967 

1 

3,450 

2 

2,190 

10,000 

13,000 

12,H5 

3 

18,550 

26,150 

26,765 

4 

40, 369 

85,000 

80,000 

85,593 

5 

48,000 

47,000 

50,000 

6 

5,000 

50,000 

45,000 

21,000 

7 

21,146 

30,000 

37,200 

45,000 

8 

2,000 

2,500 

2,500 

9 

20,000 

20,716 

20,000 

10 

1,300 

1,500 

1,000 

Table  3  -  Volumes  (Bound  serials) 


LIBRARY 

NO. 

BROOKINGS 

ATLIS  1 

FLC 

FACTSHEET 

VISIT 

1959 

1964 

1965 

1966 

1967 

1 

11 

5,900 

1 1  titles 

2 

10,000 

3 

120,000 

4 

10,700 

26,000 

26,000 

28, 085 

5 

25,000 

20, 000 

6 

15,000 

25,000 

25,000 

7 

7,190 

13,000 

400  titles 

2,003 

8 

500 

35  titles 

9 

18,000 

10 

Table  4  -  Serial  Subscr 


LIBRARY  NO. I  BROOKINGS  ATLIS  I 

1959  1964 


FACTSHEET  ' 
1966 

! 

-  < 

M  -* 

375 

} 

1 

* 

471  titles 
726 

! 

1,°71  j 

! 

i 

i 

4,559 

j 

4,150 

j 

1,000 

1,000 

1 

1,100  I 

1,000 

1,200 

: 

3 

1,200 

! 

95 

105 

1 

156 

i 

l 

1 75  titles 
250 

100 

125  titles 

Table  5  -  Pamphlets,  Reports  and/or  Documents 


BROOKINGS 

1959 

ATLIS  1 
1964 

7,p00 

8,000 

12,650 

608,800 

120,000 

740,000 

300,000 

30,000 

50,000 

67,000 

40,000 

6,100 

58,000 

7,000 

FACTSHEET 


1965 


700,000 


10,000 
660,000 
303, 742 
330,000 
46,000 

6,200 

20,000 

8,000 


VISIT 

1967 


11,000 

311,782 

360,000 

46,000 

30,000 

6,200 

12,  500 


LIBRARY  NO.  BROOKINGS 
1959 


ATUS  1 

FLC 

FACTSHEET 

VISIT 

1964 

1965 

1966 

1967 

8,252 

27,303 

34, 323 

440,000 

507,344 

571,963 

10,000 

15,000 

500 

500 

10,000 

10,000 

2,350 

TABLE  7 


Summary  of  Observations  on  Weeding  Policies* 


Library 

Number 

Brookings 

CEIR 

Visit 

1 

X 

X 

2 

X 

X 

X 

3 

X 

X 

X 

4 

X 

X 

X 

5 

X 

6 

X 

X 

X 

7 

X 

X 

X 

8 

X 

9 

X 

X 

10 

X 

* 

X  * s  indicate  performance 

of  weeding 
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Activities 


ccnstancv  becomes  apparent.  This  probably  should  have  been  expected 
in  view  of  the  fact  that  as  far  as  staff  size  and  composition  and  col¬ 
lection  size  and  composition  are  concerned,  the  situation  is  dynamic. 

The  figures  are  marked  by  change  rather  than  stability.  Thus,  it  is 
difficult  to  Judge  the  accuracy  of  the  data, even  with  on-site  inspection. 
The  dates  of  collection  and  the  manner  of  collecting,  reporting  and 
interpreting  the  data  can  cause  variations.  The  values  shown  in  the  tables 
bear  out  this  observation. 

The  effort  to  validate  the  data  presented  in  the  CEIR  report  during 
our  visits  was  difficult.  Most  of  the  people  interviewed  were  vap.ue  about 
what  they  had  reported  in  the  prior  survey.  Even  when  the  data  reported 
for  them  were  shown  again  to  them,  there  was  considerable  ambiguity  and 
few  were  either  willing  or  able  to  provide  revisions.  It  was  also  noted 
that,  although  one  of  the  libraries  had  completely  related  activities  it 
was  performing,  the  study  did  not  show  this. 

One  of  the  anomalies  interesting  to  note  was  that  some  of  tha  smaller 
libraries,  with  less  professional  staff  and  less  complete  collect  ons 
and  facilities,  checked  more  activities  as  being  performed  by  their  library 
than  some  of  the  larger,  much  more  complete  and  professionally  staffed 
organizations.  This  would  suggest  that  there  was  considerable  latitude 
in  interpreting  questions  as  to  function  definition  and  performance.  It 
was  also  found  that  even  on-site  visits  could  not  validate  all  thn  claimed 
activities  and  functions.  However,  it  is  reasonable  to  state  that  it  was 
easier  to  validate  the  information  on  libraries  which  had  claimed  fewer 
activities  and  functions  than  those  which  had  claimed  a  higher  number. 

As  a  check  on  validating  some  of  the  policy  statements,  the  example 
of  "weeding"  was  chosen.  As  can  be  seen  in  Figure  2,  almost  all  libraries 
profess  to  be  performing  weeding.  The  investigators  could  not,  however, 
determine  how  extensive  the  weeding  performed  actually  is.  Furthermore, 
this  particular  policy  matter,  as  with  moat  others,  was  not  a  formal, 
written  matter,  but  rather  was  Informal  and  unwritten. 

Data  Correlation  Considerations 


In  discussing  the  analysis  of  the  data  to  determine  the  presence  or 
absence  of  correlations  between  such  factors  as  staff  size,  document 
collections,  number  of  searches  and  similar  elements,  the  investigators 
wish  to  develop  the  situation  in  two  steps:  A.  Determination  of  rhe 
population  and  a,pects  for  which  correlations  will  be  developed;  and  B. 
Usefulness  '■*  ■  • '  'lions  devel criteria  for  measurement  of  ef¬ 
ficiency  ana  1  -  clweness .  The  following  sections  deveion  concepts 

for  sample  cases.  It  should  be  noted  that  the  details  are  not  included 
in, this  report  for  all  the  correlations  attempted,  but  are  available  if 
required.  Should  the  methods  used  prove  valuable  In  criteria  development, 
the  details  for  given  cases  can  then  be  utilized  in  later  stages  not  in¬ 
cluded  in  the  present  contract  work,  but  more  properly  to  be  done  under 
a  separate  arrangement  by  a  contractor  or  performed  by  an  Army  in-house 
\  effort. 
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A.  Determinat Ion  of  the  population  and  aspects  for  which  correla¬ 
tions  will  be  developed. 

The  population  under  study  for  the  development  of  tentative  criteria 
Include  Army  Technical  Libraries  which  have  submitted  data  and  information 
which  will  permit  computations  and  determinations  of  certain  library  para¬ 
meters  in  regards  to  staff  size  and  positions;  and  the  number  of  books, 
documents  and  technical  reports,  subscriptions  and  microforms  in  the  col¬ 
lection.  The  data  were  statistically  interpreted  to  determine  the  distribu¬ 
tion  of  the  population.  Sixty-one  libraries  constitute  the  population  in 
this  aspect  of  the  study.  The  population  identified  is  not  homogeneous. 

This  implies  that  subpopulations  can  be  identified. 

A  histogram  was  used  to  show  the  distribution  of  the  various  parameters. 
Five  histograms  are  represented  in  Figures  3,  4,  5,  6  and  8.  There  are  61 
blocks  on  each  histogram,  one  for  each  library.  The  blocks  filled  in  with 
cross-hatching  are  those  libraries  which  were  visited  by  our  team.  Although 
10  libraries  were  visited,  not  all  of  the  data  needed  was  provided  by  each 
library.  Therefore,  some  of  the  charts  and  tables  will  present  data  for 

fewer  than  10  libraries.  The  data  from  these  libraries  will  be  statistically 

manipulated  to  draw  Inferences  about  the  whole  population  of  Army  Technical 
Libraries . 

The  distribution  of  staff  size  among  the  population  studied  is  shown 
in  Figure  3.  This  histogram  identifies  15  classes  of  libraries  according 
to  staff  size.  The  classes  are: 

Class  1  —  libraries  having  1  to  4  staff  members; 

Class  2  —  libraries  having  4  to  7  staff  members,  etc. 

The  histogram  shows  a  skewed  distribution  in  terms  of  staff  size,  which 
is  characteristic  of  crowding  at  one  end  and  a  gradual  taper  to  the  op¬ 
posite  end  of  a  scale. 

The  distribution  in  Figure  3  shows  crowding  of  libraries  in  the  smallest 
staff  size  class  and  a  sharp  drop  over  the  next  two  classes.  There  Is  a 
gradual  drop  over  the  next  12  classes.  This  population  distribution  shows 
a  trend  which  is  reflected  by  the  sample. 

Skewed  distributions  were  also  found  for  the  number  of  documents 
and  technical  reports  in  library  collections.  Figure  4,  and  the  number  of 
documents,  technical  reports  and  books  in  the  collection.  Figure  5.  There 
does  not  appear  to  be  any  trend  in  the  population  in  terms  of  the  percent: 
of  books  in  library  collections  as  shown  in  Figure  6.  In  all  cases  the 
sample  reflected  the  population  trends.  However,  Figures  3  through  6  in¬ 
dicate  that  the  sample  means  do  not  always  closely  estimate  the  population 
means.  The  fact  that  population  means  are  usually  lower  than  sample  means 
shows  that  the  sample  is  biased  in  that  the  larger  libraries  had  more  thsn 
an  equal  chance  of  being  represented  in  the  sample.  This  bias  occurs  du-  to 
criteria  5,  9  and  10  p.16  and  p.  17.  The  conclusions  drawn  in  this  anal  sis 
will  take  the  bias  into  account. 
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F1GUM  6.  POPULATION  HISTOGRAM  OF  S  SOCKS  IN  COLLECTION 
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FIGURE  7.  CORRELATION  OF  STAFF  SIZE  VS.  FfcMiU  Gf  '.OCLMZMS,  TECHNICAL  REPORTS  &  BOOKS  IN  COLLECTION 


When  data  classes  with  similar  trends  such  as  number  of  staff  and 
number  of  documents,  technical  reports  and  books  in  the  collection  are 
correlated,  ve  may  see  a  relationship  between  the  two.  Figure  7  is  a  plot 
of  the  lihrarles  in  the  population  in  terms  of  staff  size  vs.  the  number 
of  documents,  technical  reports  and  books  in  the  collections.  The  libraries 
visited  are  identified  by  the  circled  points.  The  circled  numbers  refer 
to  the  library  numbers  in  Tables  1  through  7.  A  line  estimated  best  to  fit 
the  9  circled  points  is  shown  on  the  chart  (Figure  7).  The  libraries  are 
expected  to  be  scattered  about  this  line  in  normal  distribution.  This  meens 
that  68Z  of  the  libraries  are  expected  to  lie  within  one  standard  deviation 
from  this  line  and  95X  of  the  libraries  are  expected  to  lie  within  two 
standard  deviations  from  this  line.  Standard  deviations  are  both  plus  and 
minus  values  from  the  line  of  best  fit. 

One  standard  deviation  (1  S.D.)  for  the  sample  libraries  was  computed 
to  be  39.6  units  from  the  line  of  best  fit,  on  an  arbitrary  scale.  The 
one  standard  deviation  lines  are  represented  by  the  dashed  lines,  called  68Z 
confidence  limits.  The  two  standard  deviations  are  represented  by  the 
dashed  lines,  called  95%  confidence  limits.  The  sample  statistically  pre¬ 
dicts  that  68Z  of  all  libraries  will  fall  with  +  1  S.D.  and  that  95X  of  the 
libraries  fall  with  +  2  S.D.,  assuming  that  the  population  is  normal  and 
that  the  sample  is  representative. 

A  histogram  was  made  of  the  deviations  from  the  estimated  line  of  best 
fit  of  the  sample.  This  histogram  (Figure  8)  shows  that  the  total  population 
approaches  a  normal  population.  A  normal  population  is  recognized  by  a 
characteristic  bell  shaped  curve  similar  to  the  dashed  curve  in  Figure  8. 

The  actual  mean  deviation  of  the  population  from  the  line  of  best  fit  was 
computed  to  be  0.5  below  the  line  of  best  fit.  This  calculation  used  the 
same  arbitrary  scale  as  was  used  for  the  sample  deviations.  Figure  8  shows 
the  sample  mean  and  the  actual  population  mean.  It  can  be  observed  that 
the  error  in  the  estimate  was  negligible. 

The  sample  predicts  that  68Z  of  the  libraries  in  the  population  fall 

within  +  1  S.D.  A  count  of  the  libraries  within  +  1  S.D.  shows  that  actually 

77%  of  the  points  fall  within  this  range.  The  error  in  the  estimate  is  13. 31. 
The  sample  also  predicts  that  95Z  of  the  libraries  in  the  population  fall 

within  +  2  S.D.  A  count  of  the  libraries  within  +  2  S.D.  shows  that  actually 

90. 2Z  of  the  libraries  fall  within  this  ...  71."  error  in  the  estimate 

* s  5.3Z. 


Therefore,  reliable  (less  than  15Z  error)  correlations  can  be  drawn 
between  the  data  classes  collected  from  the  sample  of  libraries.  These 
correlations  can  be  assigned  confidence  limits  and  each  library  can  be 
classified  according  to  its  deviation  from  the  norm.  Any  library  within 
+  1  S.D.  can  be  classified  as  a  group  C  library.  Any  library  between  +1  S.D. 
and  +  2  S.D.  can  be  classified  as  a  group  D  library.  Any  of  the  libraries 
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higher  than  +2  S.D.  can  be  classified  as  group  E  libraries.  Any  library 
between  -1  S.D.  and  -2  S.D.  can  be  classified  as  a  group  B  library.  Any 
library  lower  than  -2  S.D.  can  be  classified  as  a  group  A  library. 

Since  there  nay  be  errors  in  the  sample's  predictions  of  the  parameters 
of  the  population,  those  libraries  which  are  close  to  the  margin  between 
groups  cannot  be  classified  with  high  confidence. 

Correlation  analyses  were  conducted  between  a  variety  of  data  classes 
where  correlations  may  be  meaningful.  The  line  of  best  fit  method  was 
used  for  these  analyses. 

Significant  positive  correlations  were  found  between  the  following 
classes  of  data  by  visual  inspection  of  scatter  diagrams. 

a.  The  number  of  items  acquired  per  week  and 

1.  Cost  of  acquisitions  per  week. 

2.  dumber  of  titles  cataloged  and  classified  per  week. 

3.  Cost  of  cataloging  and  classifying  per  week. 

4.  Cost  of  preparation  and  maintenance  of  materials  per  week. 

b.  The  number  of  dociments  and  periodicals  acquired  per  user  per 
,  week  and  items  circulated  upon  request  per  user  per  week. 

c.  The  number  of  titles  cataloged  and  classified  per  week  and  the 
cost  of  cataloging  and  classifying  per  week. 

d.  The  circulation  of  items  upon  request  per  week  and 

1.  Cost  of  circulation  upon  request  per  week. 

2.  Number  of  documents,  reports,  theses  and  bound  volumes 
per  user. 

e.  The  circulation  of  Items  upon  request  per  user  per  week  and  the 
number  of  items  circulated  by  predetermined  routing  per  user 
per  week. 

f.  The  number  of  items  circulated  by  predetermined  routing  per  week 
and 

1.  Cost  of  predetermlaed  routing  per  week. 

2.  The  number  of  subscriptions  In  thu  collection  per  user. 

g.  Total  circulation  of  items  per  user  per  week  and  the  number  of 
documents,  reports,  theses,  bound  volumes  and  subscriptions 
per  user. 

h.  The  number  of  reference  searches  per  week  and  the  cost  ->f 
reference  searches  per  week. 


i.  The  number  of  bibliographic  citations  compiled  per  week  and 
the  cjst  to  compile  these  references  per  week. 

J.  The  number  of  documents,  technical  reports  and  Looks  Ir.  the 
collection  and  the  staff  size. 


Correlations  we  -e  tried  between  the  following  classes  of  data  ard  no 
significant  or  meaningful  correlations  were  found: 


a.  The  number  of  items  acquired  per  week  and 

1.  The  number  of  items  circulated  upon  request  per  user  per  week. 

2.  The  total  number  of  items  circulated  per  user  per  week. 

b.  The  number  of  items  circulated  upon  request  per  week  ard  the 
total  number  of  actual  users.* 


c  The  number  of  items  circulated  upon  request  per  user  per  week  and 

1.  Percent  of  B.S.  degrees  among  users. 

2.  Percent  of  M.S.  degrees  among  users. 

3.  Percent  of  Ph.D.  degrees  among  users. 

d.  The  total  number  of  actual  users*  and 


1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 
9. 

10. 


The  staff  size. 

The  number  of  documents,  reports  and  theses  ir.  the  collection. 
The  number  of  volumes  in  the  collection. 

The  number  of  titles  in  the  collection. 

The  number  of  periodical  titles  in  the  collection. 

The  number  of  bov'nd  journals  in  the  collection. 

The  number  of  journals  in  the  collection. 

The  number  of  subscriptions  in  the  collection. 

The  number  of  microforms  in  the  collection. 

The  total  number  of  items  in  the  collection. 


e.  The  number  of  documents,  reports,  theses  &  bound  volumes  per  user  and 


1.  The  number  of  items  circulated  by  interlibrary  loa*  to  the 
library  per  week. 

2.  The  number  of  items  circulated  by  interlibrary  loan  from  the 
library  per  week. 


B.  Usefulness  of  correlations  in  developing  criteria  for  measurement 
of  efficiency  and  effectiveness. 


Note:  The  totel  number  of  actual  users  does  not  seem  to  correlate  highly 
with  any  of  the  parameters  tested.  This  can  be  partially  accounted 
for  by  the  high  variability  in  the  average  frequency  of  use  by 
identified  users  of  variant  libraries. 
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The  data  classes  Identified  in  Section  A  which  had  meaningful  cor¬ 
relations  with  other  5ata  classes  are  potential  areaa  for  development  of 
standards  (4,310).  These  standards,  if  developed  after  Phase  III 
of  this  contract,  could  be  used  for  measuring  efficiency  of  library 
operations.  For  example,  if  there  is  a  close  linear  correlation  between 
cost  of  cataloging  and  classifying  and  the  number  of  titles  cataloged  and 
classified,  a  standard  unit  cost  may  be  derived  (20,  125,  11,  51,  74,  118, 
382).  If  there  is  a  close  curvilinear  correlation  between  cost  and  number 
of  titles  cataloged,  then  a  conditional  standard  can  be  derived.  This  can 
be  in  the  form  of  cost  ~  some  function  of  the  nun«her  of  titles  cataloged  and 
classified.  Such  standards  could  be  used  to  indicate  relative  efficiency 
among  libraries. 

If  unit  cost  of  cataloging  and  classifying  are  above  the  standard  cost, 
we  would  expect  effectiveness  to  be  higher  or  would  expect  some  other  just¬ 
ification  for  higher  costs  such  as  higher  wage  rates.  If  it  cannot  be  shown 
that  effectiveness  is  higher,  or  that  wage  rates,  etc.  extenuate  deviation 
from  the  standard,  the  extra  cost  cannot  be  justified.  If  reliable  effective¬ 
ness  measures  can  be  determined,  deviations  from  the  norm  may  be  justified 
when  additional  effectiveness  is  realized  from  higher  unit  costs. 

In  order  to  have  useful  and  enforceable  standards,  not  only  must  the 
standards  be  realistic  and  attainable,  but  also  we  must  be  able  to  justify 
or  reject  any  costly  deviations  from  the  standard. 

Thic  report  later  discusses  effectiveness  measures  in  Type  4  and  5 
approaches.  After  some  of  these  measures  are  taken  in  Phase  III  of  this 
contract,  correlations  between  effectiveness  and  unit  costs  may  be  determined. 
If  high  correlations  exist,  criteria  can  be  developed  which  relate  operation 
output  or  unit  costs  of  operations  to  effectiveness  (141,  155).  Correc¬ 
tions  such  as  that  described  in  Figure  1  p.  12  were  intended  for  considera¬ 
tion  under  Type  3  approach.  However,  some  criteria  developed  in  Type  4 
approach  must  first  be  validated.  Therefore,  these  correlations  and  further 
analysis  will  be  performed  in  Phase  III. 

Standards  developed  by  correlations  are  potentially  useful.  The  data 
ao  far  gathered  reflect  that  the  standards  would  be  ’’loose."  That  is  to 
say,  such  standards  would  be  based  on  averages  and  may  in  some  cases  be 
standards  of  mediocrity.  However  they  would,  nonetheless , be  points  from 
which  to  measure.  This  looseness  is  due  to  a  number  of  possible  factors, 
including: 

1.  Lack  of  standard  operational  procedures  among  the  Army  Technical 
Libraries . 

2.  Lack  of  consideration  of  quality  and  effectiveness  of  the  work 
units  produced. 

3.  Lack  of  clearly  defined  operations  and  services. 

4.  Variability  of  wage  rates  for  similar  operations  among  and 
within  the  Army  Technical  Libraries. 

5.  Variability  of  efficiency  in  operational  procedures  among 
and  within  the  Army  Technical  Libraries. 

6.  Errors  in  estimates  of  work  performed,  costs,  number  of  users, 
number  of  volumes,  etc. 
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In  order  to  develop  "tighter"  and  sore  useful  standards,  it  will  be 
necessary  to  quantify  or  eliminate  some  or  a '  cf  these  farters. 

In  an  effort  tc  demonstrate  that  these  factors  can  be  quantified  or 
eliminated  by  the  use  of  selected  management  techniques,  Phase  III  of 
this  contract  will  be  dedicated  primarily  to  developing  Group  Attainment 
Program,  cost  effectiveness  analysis  and  utility  analysis.  The  choice  of 
these  l.  chniques  is  discussed  in  Chapter  VII,  Type  4  approach. 

Factors  1,  2  and  3  can  be  quantified  or  eliminated  in  utility  analysis 
or  cost-effectiveness  analysis  utilizing  a  "systems"  approach  through 
modeling  to  define  boundaries  of  operations  and  services  and  to  identify 
the  activities,  costs,  objectives  and  effectiveness  of  services  (768). 

Factors  4,  5  and  6  can  be  taken  into  account  through  Group  Attain¬ 
ment  Program  utilizing  work-sampling  studies  on  some  routine  operations. 

After  standards  of  efficiency  and  measures  of  effectiveness  are 
established  by  these  management  techniques,  criteria  may  be  developed  from 
correlations  which  relate  standard  unit  costs  of  operations  to  effectiveness. 
Standard  unit  costs  are  total  unit  costs  minus  avoidable  costs  per  unit. 
Avoidable  costs  inc  ude  costs  due  to  slow  work  pace  ar.d  costs  due  to  work 
delays  in  excess  of  allowances. 


VII.  TYPE  4  APPROACH  EVALUATION  AND  ANALYSIS 


This  approach  explores  the  possibility  of  developing  criteria 
from  various  "Management  Techniques.”  These  techniques  were  presented 
in  ATUS  REPORT  NO.  10,  CRITERIA  FOR  EVALUATING  THE  EFFECTIVENESS  OF 
LIBRARY  ui-ERaIIONS  aND  SERVICES.  FKA3  C  I:  LITETATULF  SEARCH  AND  STATE 
OF  THE  ART.  AD-649  468.  These  techniques  were  presented  in  the  for®  of 
a  matrix.  The  definitions  were  giver,  in  an  appendix  and  discussed  through¬ 
out  the  text.  For  example,  reference  to  the  matrix  shows  that  "cost 
effectiveness"  was  judged  applicable  for  studyiro  ”biv 1 iographic  services.” 
The  data  collected  for  a  cos.  effectiveness  assessment  of  this  service  may 
permit  the  development  of  criteria  statements.  The.  kind  of  data  required 
fo?  this  approach  included  fa)  statistics  on  library  work  measurement 
programs;  (b)  operating  cost  data  such  as  weekly  costs  of  acquisitions  and 
accessions,  cataloging  and  classifying  and  circulation  activities;  (c) 
operations  output  date  such  as  the  weekly  number  of  titles  cataloged  and 
classified,  number  of  units  (books,  documents,  reports)  acquired;  (d) 
budget  data;  (e)  layout,  charts;  (f)  ether  statistics  on  the  library  such 
as  number  af  items  (books,  documents,  reports),  staff  size  ar.d  grades, 
number  of  actual  and  potential  users  and  (g>  standard  data.  Other  in¬ 
formation  of  this  type  concerned:  { e)  organizational  facts  such  as  mission? 
structure,  and  distribution  of  functions,  responsibilities  ana  control: 

(b)  methods  of  library  performance  evaluation;  tc)  value  of  user  surveys; 
(d)  validity  of  various  measures  of  effectiveness;  and  (e)  methods  and 
equipment  to  perform  operations  and  services. 

Because  there  are  so  many  techniques  available,  it  was  thought 
reasonable  to  reduce  the  number  to  a  few  for  actual  consideration.  Die 
techniques  listed  in  the  matrix  of  the  Phase  I  Report  were,  therefore, 
analyzed  for  their  usefulness  in  indicating  efficiency  and/or  effectiveness 
or  for  controlling  performance  of  certain  library  operations  and  services. 
Before  l.iis  was  done,  however,  the  library  functions  were  classified  ac¬ 
cording  to  the  contractor's  concepts  of  high  and  low  priority. 

The  analytical  method  employed  fer  selecting  management  techniques 
for  further  consideration  in  the  contract  is  the  "paired  comparison"  method 
Three  consultants  on  the  contractor's  staff,  trained  and  experienced  in 
management  techniques,  performed  analyses  independently.  Each  was  supplied 
with  definitions  of  the  library  functions  and  the  glossary  of  the  manage¬ 
ment  techniques  presented  in  the  Phase  I  Report  with  some  revisions  ana  new 
considerations.  The  three  consultants  were  instructed  identically  as  to 
the  methods  they  were  to  use  in  making  the  comparisons.  They  were  not 
informed  .about  the  priority  ratings  assigned  to  the  functions. 

In  actual  use,  the  paired  comparison  method  employed  a  type  chart 
for  each  individual  library  function.  Figure  9  illustrates  the  method, 
in  this  particular  case  used  for  selecting  management  techniques  for 
evaluating  circulation  service.  All  management  techniques  identified 


60 


PGUtC  f.  C'*CUL»nON  SEJMCE 


either  in  Phase  Report  or  added  during  the  Phase  II  work.,  for  the 
library  function  under  consideration,  were  reviewed.  Those  chosen  as 
potentially  useful  for  circulation  service  evaluation  or  control  are 
listed  in  both  she  left-hand  vertical  column  and  the  horizontal  top  row 
of  the  chart.  Although  the  title  of  the  technique  is  provided  only  in 
the  left-hand  vertical  column,  the  Roman  numerals  used  in  both  columns 
and  rows  indicate  identical  techniques.  The  references  shown  in  the  right- 
hand  vertical  column  represent  abstracts  in  the  Phase  I  Report,  or  pub¬ 
lications  identified  in  the  Supplemental  Management  Science  References  ir. 
the  present  report . 

In  the  second  left-hand  vertical  column,  3  comparison  factors  are 
repeated  for  eacn  technique  compared: 

1.  Indicator  of  efficiency; 

2.  Indicator  of  effectiveness: 

3.  Controller  of  performance. 

Each  management  technicue  can  thus  be  compar_d  with  all  other  tech¬ 
niques  listed  for  that  particular  library  function.  The  judgments  of  the 
management  consultants  who  performed  the  comparisons  were  indicated  by  ’ye®5' 
or  "no"  notations  in  the  spaces  provided  on  the  charts.  A  "yes"  notation 
indicates  that  the  technique  in  the  vertical  column  is  a  better  indicator 
or  controller  than  the  technique  with  which  it  is  being  compared  in  the 

top  horizontal  row  for  the  particular  library  function.  An  arrow  indicate* 

that  the  technique  in  the  vertical  column  is  or  can  be  a  subtechnique  of 
the  more  generic  technique  represented  in  the  top  horizontal  row. 

After  ail  judgments  were  performed,  the  "yes"  notations  were  totaled 
to  obtain  a  rating  figure  for  each  technique.  These  ratings  were  then 
converted  to  ranks  for  each  technique  on  each  comparison  factor.  The  ranks 

among  the  3  judges  were  averaged  to  determine  the  final  rank  for  each  tech¬ 

nique.  The  highest  ranked  techniques  for  each  library  function  on  each 
comparison  factor  are  summarized  in  Figures  10  and  11. 

As  will  be  recalled  from  the  Phase  I  Report,  certain  management  tech¬ 
niques  are  referred  to  usually  as  "analytical"  and  others  as  "operative." 
Thus,  those  techniques  or  tools  assigned  the  letter  "A"  in  the  bottom  rows 
of  Figures  10  and  11  are  analytical,  and  can  be  used  ro  study,  develop  and 
justify  criteria.  Those  tools  assigned  the  letter  ”0"  are  regarded  as 
operative  techniques  which,  if  applicable,  can  be  used  to  implement  criteria 
or  develop  standards  of  effectiveness  or  efficiency  of  library  functions. 

Due  to  time  limitations  on  this  particular  data  type  effort,  only  3 
techniques  will  be  used  as  examples  to  develop  criteria.  These  are  GAP 
(Group  Attainment  Program),  Cost-Effectiveness  Analysis  and  P.P.B.S. 
(Pianning-^rogramming-Uudgeting  System),  These  techniques  are  to  be  ur>ed 
formally  in  developing  criteria  as  discussed  in  subsequent  sections.  It 
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should  be  kept  in  mind,  however,  that  elements  of  several  other  manage¬ 
ment  techniques  are  used  in  other  sections  of  this  work  and  report,  but 
in  a  more  informal  sense.  For  example,  systems  analysis  and  utility 
analysis  techniques  are  uspd  freely,  probability  .v^thods  are  involved, 
correlations  are  made,  and  elements  of  cost-accounting,  models,  charting 
techniques,  personnel  administration,  management  analysis,  and  several 
others  are  touched  upon.  GAP  and  Cost-Effectiveness  techniques  will, 
however,  be  discussed  in  some  detail. 

METHOD  OF  DEVELOPING  CRITERIA  WITH  OPERATIVE  TECHNIQUES 

Group  Attainment  Program  (see  GAP  XVI  in  Figure  10)  was  judged  the 
most  useful  tool  for  indicating  efficiency  of  routine  library  operations, 
such  as  cataloging  and  classifying,  abstracting,  indexing,  acquisitions, 
preparation  and  maintenance  of  library  materials,  and  overall  staff  utiliza¬ 
tion.  Using  proven  measurement  techniques  such  as  work  sampling,  standard 
data  and  motion  and  time  studv,  GAP  objectively  determines  attainment 
factors  for  the  jobs  in  a  group.  These  attainment  factors  represent  the 
time  that  it  should  take  a  competent  employee,  using  prescribed  methods 
and  working  at  a  normal  pace,  to  perform  one  operation  successfully  and 
completely.  Having  determined  the  "should  take"  time  for  all  operations, 
it  is  aDplied  to  present  or  planned  volumes  of  operations  to  determine 
total  labor  requirements  for  the  entire  volume  of  work.  Attainment  factors 
or  "should  take"  times  are  recorded  as  specific  standard  data  (Standard 
Data,  XIII  in  Appendix  A) . 

GAP  indicates  efficiency  of  groups  hv  comparing  actual  time  spent 
to  "should  take"  time.  The  measure  of  efficiency  is  expressed  as  the 
index  of  productivity.  This  index  is  computed  bv  dividing  "should  take" 
time  by  actual  time.  An  index  of  productivity  can  he  computed  for  each 
group  performing  given  operations  or  for  the  whole  staff  giving  an  over¬ 
all  measure  of  staff  utillzat  1,  The  operative  criterion  of  GAP  is  its 
end  product — unit  cost  or  unit  time  standards  for  each  significant 
operation, 

METHODS  OF  DEVELOPING  CRITERIA  WITH  ANALYTICAL  TECHNIQUES 

Cost-Effectiveness  Analysis  (Cost  Effectiveness,  II,  Figure  10),  has 
been  judged  the  best  technique  for  indicating  efficiency  and  effectiveness 
of  typical  library  services,  such  as  circulation  service,  information 
retrieval  service,  translation  service,  bibliographic  service  and  reference 
service.  Cost-Effectiveness  Analysis  measures  cost  and  ef t ectiveness  of 
each  service  to  provide  a  means  of  judging  overall  performance  of  each 
service  or  for  *11  services.  For  example,  let  us  make  the  assumption  that 
there  are  3  possible  methods  for  providing  predetermined  circulation, 
methods  #1,  H 2  and  l?3.  Hypothetically,  the  costs  are  $1.00,  $1.50  and 
$2.00,  respectively,  per  item  circulated,  and  the  effectiveness  measures 
of  meeting  the  objectives  of  the  service  are  501,  UQZ  and  702,  respectively 
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A  plot  the  methods  represents  a  cost-effectiveness  diagram  (Figure  12) 
and  shows  the  relative  cost-effectiveness  of  each  method.  The  choices 
of  methods  which  appear  most  practical  are  circled.  Method  #2  is  im¬ 
practical  because  it  is  more  costly  and  less  effective  than  method  #1. 

The  two  choice;  left  are  evaluated  in  terms  of  the  librarian's  preference: 
being  20Z  more  effective  or  spending  only  SI. 00  per  item  circulated. 

The  library  should  have  a  well  developed  mission  statement  and  budget  to 
facilitate  deciding  the  most  practical  choice  between  methods. 

In  order  to  make  a  Cost-Effectiveness  Analysis  of  services  possible, 
it  is  necessary  to:  (1)  define  each  significant  service,  (2)  define  all 
significant  activities  and  operations,  (3)  provide  meaningful  cost  data 
for  each  activity  and  operation,  (4)  provide  meaningful  effectiveness  data 
for  each  operation  and  service.  After  this  has  been  accomplished,  criteria 
for  effectiveness  of  operations  and  services  can  be  derived.  For  example, 
the  extent  and  efficiency  of  cataloging  and  classifying  may  show  a  cor- 
relatlun  with  ef fecclvcnest,  of  certain  services.  Such  criteria  can  be  drawn 
from  real  world  situations  at  each  liv'rary.  The  task  at  hand,  therefore, 
is  to  simulate  real  world  situations  as  closely  as  possible.  In  doing  so, 
the  criteria  are  expected  to  become  apparent  (Phase  I  Report,  References 
4,  14,  23,  46,  51,  55,  310,  410,  415;  this  report,  References  766,  767). 

Such  a  simulation  has  been  attempted  by  a  generalized  model  of  services 
and  operations  in  a  ty  leal  Amy  Technical  Library.  This  has  been  done 
by  representing  real  world  situations  in  chains  of  events  joined  by 
activities  performed  to  actuate  each  progressive  event.  The  simulation  of 
4  typical  services  is  shown  in  Figures  13,  14,  15  and  16. 

Each  service  chain  has  one  or  more  entry  points.  All  entries  from 
event  1  are  origins  of  library  service  needs.  All  entries  from  other 
events  are  continuations  of  needs  which  have  required  previous  library 
services.  In  test  cases  in  Phase  III,  or  in  actual  use,  event  blocks 
critical  to  service  effectiveness  will  be  assigned  probabilities  that  the 
event  will  occur.  These  probabilities  will  be  based  upon  past  experience 
using  statistical  and/or  subjective  analysis  or  by  statistical  sampling. 

Each  activity  is  represented  by  the  connecting  line  between  events.  Ac¬ 
tivities  which  accrue  significant  costs  will  be  assigned  standard  cost  data. 

These  standard  cost  data  will  be  derived  from  past  experience  or  can  be 
generated  by  work  sampling,  motion  and  time  studv,  GAP  and  cost  accounting 
techniques.  The  standard  cost  data  will  be  expressed  as  average  unit  cost 
per  need  or  as  expected  cost  per  100  needs  or  some  similar  representation 
of  coat  standardization. 

Measures  of  efficiency  mav  the  standard  cost  data  from  which  an 
index  of  productivity  can  be  calculated  for  given  work  periods.  Measures 
of  effectiveness  mav  be  derived  from  factors  which  influence  the  probabil¬ 
ities  that  events  will  occur  when  a  number  of  needs  are  processed. 
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FIGURE  12  -  COST  EFFECTIVENESS  CHART 
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Certain  event  blocks  separate  and  identify  factors  which  influence 
effectiveness.  For  example,  In  Figure  13  event  3  ( the  librarian  begins 
a  search),  two  possible  reasons  which  could  explain  why  a  reference 
librarian  would  not  process  a  need  further  are  (a)  the  'ed  is  not  within 
the  scope  of  the  library's  mission  and  (b)  the  priority  of  other  rasks. 
Indicators  of  the  percent  cause  of  failure  to  accomplish  the  event 
represented  by  subprobsbilitles  will  help  identify  the  reason  for  low 
probabilities  of  effectiveness.  These  causal  identifiers  can  Indicate 
criteria  for  effectiveness.  Effectiveness  causal  factors  may  be  any 
identifiers  which  reflect  a  responsiveness  of  effectiveness  to  the  extent 
of  the  factor  present.  (Reference  768,  p.13) 

Another  effectiveness  causal  factor  is  the  direct  labor  operation 
preceding  each  event  block.  For  example,  in  '.‘igure  13,  event  4  (some 
candidate  documents  are  identified)  is  preceded  bv  the  activity  of  the 
librarian  conducting  a  reference  search.  This  activity  lying  between  event 
3  and  event  4  will  be  referred  to  as  activity  3-4.  The  effectiveness 
probability  of  event  4  depends  upon  the  performance  of  activity  3-4. 
Mathematically  expressed,  event  4  ■  f  (activity  3-4),  i.e.,  event  4  is  a 
function  of  activity  3-4.  The  effectiveness  of  activity  3-4  can  be  measured 
to  some  extent  bv  the  probability  of  event  4  occurring.  Conversely,  the 
occurrence  of  event  4  is  partially  dependent  upon  t.ie  time,  effort  and 
facilities  allocated  In  activity  3-4.  Therefore,  performance  of  activity 
3-4  may  be  recognized  as  a  causal  factor  Influencing  the  effectiveness 
probability  of  event  4.  The  unit  coat  per  need  of  activity  3-4  may  show 
a  correlation  with  the  effectiveness  probability  of  event  4,  Where  high 
correlation  is  found  to  exist,  the  unit  cost  of  activities  becomes  a 
causal  factor  of  effectiveness. 

Another  effectiveness  causal  factor  is  the  cost  of  indirect  library 
functions.  These  functions  mav  contribute  to  effectiveness  in  the  name 
manner  as  direct  cost  activities.  Indirect  costs  to  services  are  those  which 
contribute  to  long  range  effectiveness,  such  as  costs  of  acquisitions  and 
accessions,  preparation  and  maintenance  of  library  material,  cataloging, 
classifying,  indexing,  and  setting  up  of  systems  for  information  retrieval 
and  for  selective  dissemination  of  Information.  These  tosts  must  be  pro¬ 
rated  over  their  effective  use  period.  The  costs  assigned  in  the  indirect 
factors  blocks  (Figure  i.3)  may  be  based  on  costa  prorated  per  user,  per 
100  users,  oer  unit  time  or  othe"  units.  As  in  the  case  of  direct  activity 
performance  and  unit  costs,  indrect  function  performance  and  unit  costs 
may  show  a  correlation  with  effectiveness  probabilities.  Where  high  cor¬ 
relations  are  found  to  exist,  unit  coat  or  extent  of  indirect  functions 
can  become  a  criterion  of  effectiveness. 

The  total  coat  of  a  service  can  be  computed  by  suremlng  the  direct 
and  indirect  unit  costa.  The  total  effectiveness  of  a  service  can  be 
determined  bv  the  number  of  needs  which  meet  the  objectives  of  the  service 
relative  to  the  numbet  of  needs  entering  the  service.  Along  any  chain  of 
events  and  activities,  effectiveness  E  can  be  computed  bv  the  product  of 
the  ef f ectivenese  probabilities  or  by  the  formula: 
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E  ”  No.  of  needs  processed  through  services 1  final  objective  x  100Z 

No.  of  entry  needs 

The  intrinsic  effectiveness  of  c  service  to  a  user  nay  not  neces¬ 
sarily  be  represented  by  event  probabilities.  The  probabilities  are, 
however,  indicators  of  the  effectiveness  of  library  expenditures  in  actu¬ 
ating  essential  events.  The  ef fectiveness  of  search  service,  for  example, 
depends  on  the  objectives  of  the  service.  The  objectives  can  be  represented 
generally  by  the  possible  events  in  a  service  chain.  For  example,  one 
objective  of  reference  service  or  search  service  may  be  only  to  identify 
some  candidate  documents  for  an  informational  need.  The  effectiveness  of 
this  service  in  meeting  this  objective  can  be  measured  by  the  product  of 
probabilities  of  communication  of  needs,  of  the  librarian  beginning  a  search 
and  of  the  librarian  identifying  some  candidate  documents.  Another  objective 
of  search  service  may  be  to  determine,  perhaps  by  examination,  the  doc¬ 
uments  relevant  to  a  given  Informational  need.  The  effectiveness  of  the 
service  in  meeting  this  objective  can  be  measured  by  the  product  of  the 
probabilities  of  communication  of  needs,  of  the  librarian  beginning  a  search, 
of  the  librarian  identifying  some  candidate  documents,  of  these  documents 
being  retrieved  and  of  the  documents  containing  some  relevant  information. 

The  effectiveness  of  the  library  in  meeting  client  needs  is  predicated  upon 
the  goals  and  objectives  of  each  service  and  ultimately  upon  the  library 
mission.  Before  a  meaningful  measure  of  effectiveness  to  the  user  can  be 
derived,  it  is  necessary  to  determine  the  service  or  services  needed  by 
the  user,  for  example,  one  class  of  users  may  communicate  a  need  for  in¬ 
formation  on  a  given  subject.  To  meet  this  need  it  may  be  necessary  to 
provide  search  service  (Figure  13)  through  events  1,  2,  3,  4,  6  and  7,  and 
then  circulation  service  (Figure  16)  through  events  8,  10  and  11.  After 
event  11  has  occurred,  the  services  are  completed.  The  completion  of  each 
event  represents  progressing  in  steps  to  a  final  objective.  The  final 
objective  in  the  chain  of  events  1,  2,  3,  4,  6,  7,  8,  10  and  11  is  «vent 
11 — user  gains  possession  of  relevant  material.  The  overall  effectiveness 
probability  of  meeting  this  objective  for  a  population  of  users  is  the 
product  of  the  probabilities  of  events*  2,  3,  4,  6,  7,  8,  10  and  11.  Other 
measures  of  effectiveness  of  the  service  in  meeting  the  mission  of  the 
library  or  the  client  needs  can  be  determined  from  user  feedback  (Figure  17). 
If  the  user  reads  the  documents,  event  16  occurs,  so  that  the  next  activity 
would  be  user  feedback.  The  feedback  would  permit:  (1)  evaluating  the 
adequacy  of  the  source(s)  in  meeting  the  mission  of  the  library  (activity 
17-18),  (2)  evaluating  the  adequacy  of  the  source(s)  in  meeting  the  clients 
needs  (activity  18-19).  (3) **evaluating  the  effect  of  recall,  retrieval, 

and  relevance  upon  meeting  the  mission  and/or  meeting  the  users  informational 
need.  The  probability  of  meeting  the  mission  of  the  library  can  be  deter¬ 
mined  by  the  percent  of  needs  which  are  considered  satisfied  relative  to 
the  conditions  of  the  mission  statement.  The  mission-related  effective¬ 
ness  (EM)  of  the  service  is  the  product  of  the  probabilities  of  events  2, 

3b,  4,  6,  7,  8,  10,  11,  16  and  18.  The  effectiveness  of  the  service  to 
the  users  is  the  product  of  probabilities  2,  3,  4,  6,  7,  8,  10,  11,  16, 

18  and  19. 

*  The  probability  of  event  1  occurrence  will  not  be  included  in  the  dis¬ 
cussions  of  effectiveness  measures  in  Phase  II. 

**  The  effect  of  recall,  retrieval  and  relevance  and  the  use  of  recall  and 
relevance  ratios  as  measurements  of  effectiveness  will  be  dist ussed  in 
Phase  III  of  this  contracc. 
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The  measure  of  mission-related  effectiveness  is  not  necessarily 
a  measure  of  the  total  effectiveness  to  the  user  of  the  services  or  to 
the  parent  organization.  The  measure  of  raissi on-related  effectiveness 
will  more  closely  approximate  me  measure  of  effectiveness  to  the  user 
or  rent  organization  as  the  goals,  objectives  and  mission  of  the  library 
ap,  srh  the  goals  and  objectives  of  the  user  and  support  the  mission  of 
the  parent  organization.  Therefore,  the  importance  of  \:eli-formulated 
Mbrary  missions,  goals  and  objectives  in  deriving  a  meaningful  measure 
of  effectiveness  of  the  library  cannot  be  overemphasized. 

In  order  to  explain  further,  the  applicability  of  this  technique  to 
real  world  situations,  an  attempt  to  organize  various  cases  of  search 
service  into  type  categories,  has  led  to  the  following  breakdown  of  4 
service  types  and  12  service  cases: 


Type  I.  Unequivocally  expressed,  unequivocal  need. 


Case  I (a) . 
Case  I (b)  . 

Case  1(c) . 


Search  conducted  by  ’ibrarian. 

Search  conducted  by  uset  with  assistance 
from  a  librarian  or  library  assistance. 
Search  conducted  by  user. 


Type  II.  Equivocally  expressed,  unequivocal  need. 


Case  11(a).  Search  conducted  by  librarian. 

Case  11(b).  Search  conducted  by  user  with  assistance 
from  a  librarian. 

Case  11(c).  Search  conducted  by  user. 


Type  III.  Equivocally  expressed,  equivocal  need. 

Case  III (a).  Search  conducted  by  librarian. 

Case  111(b).  Search  conducted  by  user  with  assistance 
from  a  librarian. 

Case  III (c) .  Search  conducted  by  user. 


Type  IV.  Unequivocally  expressed,  equivocal  need. 

C«ise  IV(a).  Search  conducted  by  librarian. 

Case  IV(b).  Search  conducted  by  user  with  assistance 
from  a  librarian. 

Case  IV(c).  Search  conducted  by  user, 

E  n  of  the  12  cases  will  be  handled  differently  in  computing  costs 
and  effectiveness.  T'ue  probabl®  paths  taken  in  cases  1(a)  through  111(c)* 
are  marked  in  Figure  13,  An  example  of  au  equivocailv  expressed,  equivocal 
need,  case  Ilia  follows: 


ft 

The  paths  of  cases  IV(a‘, ,  IV(b),  and  IV(c)  are  not  marked  in  figure  f3 
because  of  the  rarity  of  thoit  occurrence. 
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Case  Ilia.  Client  contacts  a  reference  Librarian  seeking  some  information 
on  a  given  subject.  This  case  will  be  referred  to  as  an 
equivocally  expressed,  equivocal  need.  The  following  activities 
and  events  ensue  (Figure  18): 

Activity  1-2.  The  client  communicates  his  need  for  information  on  the 
effect  of  contaminants  on  ruby  lasers. 

Event  2.  The  need  is  adequately  communicated  (understood)  by  the  librarian 
to  process  the  need  further. 

Activity  2-3.  The  reference  librarian  reviews  the  need  in  order  to  determine 
if  service  will  be  given. 

Event  3.  The  reference  librarian  considers  the  need  as  within  the  mission 
of  the  library  and  as  having  significant  priority  to  begin  a 
search. 

Activity  3-4.  The  librarian  conducts  a  search. 

Event  4.  Some  candidate  documents  are  identified  by  the  librarian.  At  this 
point  (event  4),  a  request  for  a  bibliography  may  begin  activity 
4-5.  However,  the  client  may  not  require  a  bibliography,  but 
instead  may  ask  for  some  of  the  candidate  documents.  If  this  is 
the  case,  activity  4-6  begins. 

Activity  4-5  (alternate),  A  bibliography  is  prepared. 

Event  5  (alternate).  A  bibliography  is  provided  and  the  user  reviews  the 
listings. 

Activity  5-6  (alternate) .  The  user  orders  and  the  librarian  retrieves 
some  of  the  documents  listed  in  the  bibliography. 

Activity  4-6  (alternate).  The  librarian  retrieves  some  of  the  candidate 
documents . 

Event  6.  Some  of  the  candidate  documents  are  retrieved  within  the  re¬ 
quired  time. 

Activity  6-7.  The  client  reviews  the  documents  to  determine  relevance. 

Event  7.  Some  of  the  documents  are  relevant.  Event  7  *nd:i  the  search 
service.  At  this  point  the  client  may  reques  circulation 
service  and  exit  to  event  8  (Figure  16). 


If  100  similar  needs  were  processed  and  al’  needs  were  met,  the  ef¬ 
fectiveness  probabilities  of  all  events  would  _>e  100Z.  If  only  70  needs 
have  passed  ev*r,t  3  out  of  100  which  passed  event  2,  the  effectiveness 
probability  of  event  3  would  be  70Z.  This  may  indicate  that  the  mission 
statement  should  be  revised  to  encompass  more  of  the  needs  or  that  additional 
reference  librarians  are  needed  because  many  needs  could  not  be  met  due  to 
priority  of  other  tasks.  Such  needs  should  be  met  if  costs  are  not  prohibitive. 

Contamination  of  tne  measurements  of  effectiveness  may  arise  from  (1) 
poorly  developed  library  missione  statements  which  are  not  clear  or  com¬ 
patible  with  the  users'  departmental  missions,  (2)  poor  interpretation  of 
user  feedback,  and  (3)  inadequacy  of  feedback  to  express  the  true  utility 
of  ths  information  in  meeting  the  objectives  of  the  user. 

In  summary,  and  in  spite  of  the  errors  inherent  in  the  measurements 
of  *f f«ctiver.e33,  this  systematic  approach  is  potentially  useful  in  resolv¬ 
ing  the  probabilities  that  the  services  will  meet  the  needs  of  its  users 
to  the  extent  that  criteria  of  cost-effectiveness  can  be  established 
objectively. 

Planning-Programming-Budgeting  Systems  (See  P.P.B.S.,  V  in  Figure  10) 
was  judged  the  most  useful  tool  for  controlling  performance  of  budget 
allocations  for  services  and  operations. 

From  the  models  developed  by  Cost-Effectiveness  Analysis  expressing 
effectiveness  as  a  function  of  event  probabilities,  it  is  expected  that 
each  probability  can  be  expressed  as  a  function  of  the  amount  of  certain 
factors  present.  For  example,  the  probability  that  some  candidate  doc- 
usjents  are  identified  by  searching  a  card  catalog  may  be  a  function  of 
several  independent  variables,  such  as  amount  of  time  (t) ,  the  depth  of 
indexing  (I),  and  the  number  of  documents  (D)  in  the  relevant  subject 
classes.  The  relationship  between  the  dependent  variable  P  (the  prob¬ 
ability  of  finding  some  candidates)  and  the  independent  variables  t,  I  and 
0  stay  be  found  to  fit  some  general  equation,  where  P  =  f(t,I,D)  •  The 
general  equation  (mathematical  model)  which  best  represents  the  real  world 
relationships  may  be  logarithmic,  for  example:  P-  G't-b  In  t  +  c  In  I  ♦  d  In  0 
may  be  Che  general  equation  which  best  describes  the  relationships  between 
P,t,  I&D  ,  where  a,  b,  c  and  d  are  some  given  constants  and  In  t,  In  I  and 
In  D  are  the  natural  logarithms  of  t,  I  and  D,  respectively. 

With  such  general  equations  it  is  possible  to  determine  the  change  in 
P  with  the  change  in  any  independent  variable. 


For  example:  6  P  /  6  1  -  c/I 


that  is, the  change  in  P  with  respect  to  the  change  in  I  is  equal  to  c/I . 


Cost  parameters  can  be  added  to  derive  the  change  in  P  with  the  change  in 
the  cost  of  I. 

For  example: 

Let :  cost  of  I  :  Cj 

since:  Cj=  f  ID,  soy  Cj=  g(I) 

where: g(I)  is  the  exact  function 

then:  b  Cj  /  bl  -  6  g  ( I)  /  6  I 

bl  \  fc\ 

&c7/=  [V 

where  6  g  ( I )  /  6  I  =  g'  ( I ) 

f  p  _ 

The  equation  — -  = - - - —  can  be  used  to  measure  the  effect  of  funds 

SCX  (I)  [g'd)] 

allocated  to  indexing  in  increasing  the  probability  of  finding  some  candidate 
documents . 


therefore : 


&  P 


=  / 


/6  P\  / 


fe Ci  \6I/V 


or  61/  bCj  =  61  /  &  g  (I) 

rsi  =  c  _ 

L&gU)J  '  (I)[g'(I)] 


Equations  of  this  t”pe  may  prove  to  be  useful  criteria  for  budgeting 
funds.  Since  resources  should  be  added  to  operations  to  maximize  the 
probability  of  essential  event  occurrence,  then  funds  should  be  added 
where  &P  is  the  largest,  where  Ci  -  cost  of  I,  cost  of  t,  or  cost  of 
&Cj 

D,  etc.,  in  a  "j"  family  of  Independent  variables  any  one  of  which  would 
affact  event  probability. 

In  real  library  situations  it  may  be  desirable  *-o  maximize  other 
values  such  as  the  satisfaction  of  executive  officers  served  or  other  such 
values  based  upon  subjectivity.  If  values  other  than  maximization  of  total 
within-mission  needs  met  extenuate,  then  general  formulas  based  upon  these 
services  models  are  not  necessarily  valid  as  tools  for  budgeting  criteria. 
Further  discussion  on  subjective  effectiveness  values  to  be  measured  and 
maximized  is  discussed  in  Chapter  VIII  under  the  utility  analysis  approach. 
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Approach  5  is  explained  in  part  in  the  Phase  I  report  of  this 
contract,  (ATLIS  Report  No,  10),  beginning  on  p.  6  and  illustrated  by 
Figure  1  on  p.  9.  The  purpose  is  to  formalize  in  the  mind  of  the  librarian 
that  all  operations  performed  in  his  library  and  all  services  the  library 
produces  serve  the  goals  and  objectives  which  either  he  or  his  superiors 
have  set  an  implementing  the  overall  mission  of  his  library  This  Mission 
is  subservient  to  the  higher  mission  of  ^he  organization  within  which 
the  library  operates. 

Tbi;  approach,  therefore,  provides  the  format  and  procedures  which 
the  librarian  may  use  in  formalizing  (1)  his  library  mission  statement, 

(2)  the  goals  and  objectives  wh^ch  give  form  and  substance  to  that  mission, 

(3)  the  o»rticular  activities  or  services  his  library  should  produce  to 
implement  those  goals  sr.d  objectives,  and  finally  (4)  those  operations 
vhieh  he  and  his  staff  must  perform. 

On  the  basis  of  training  and  experience,  it  then  becomes  possible, 
using  the  matrix  provided,  for  Che  librarian  to  assign  weights  to  the 
various  services  which  describe  their  relative  importance  in  achieving 
the  mission,  goals  and  objectives.  The  librarian  can  then  estimate  how 
much  of  each  operation  is  necessary  to  produce  the  activities  and  services 
required.  It  is  then  possible  for  him  to  compare  this  situation  with  the 
real-life  situation  in  the  library  and  assess  the  ef fectivenes*  of  overall 
operations. 

This  approach  would  be  expected  to  produce  a  "management"  criterion. 

That  is,  the  effectiveness  of  a  given  library  is  a  ^unction  of  the 
organization  and  management  ability  of  the  head  librarian.  It  is  highly 
judgmental  and  proximate.  In  the  hands  ef  a  well-organized  head  librarian 
it  is  a  tool  which  assesses  his  own  effectiveness  and  demonstrates  to 
superiors  that  by  organizing  activities  and  services  to  optimize  mission 
fulfillment  he  is  effectively  fulfilling  his  management  responsibilities. 

Thin  approach,  or  perhaps  the  methods  and  procedures  involved,  permit 
periodic  adjustment  to  changing  situations.  That  is.  with  assignments 
of  new  tasks,  or  alteration  in  services,  or  other  changes,  the  librarian 
is  able  to  re-assess  his  situation  and  to  make  the  management  decisions 
required  to  meet  the  new  set  of  requirements.  In  that  sense  this  approach 
recognizes  that  library  situations  are  dynamic. 

This  approach  makes  a  number  of  assumptions  The  major  assumption 
is  that  a  »  *M  delineated  mission  statement  is  available  for  the  library 
organization  and  that  this  statement  supports  the  h'  ion  organiza¬ 

tion  (command,  department,  division,  research  group,  an.  on)  that  the 
library  serves.  The  subsequent  assumptions  are  based  on  that  major  assumption. 
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For  example,  the  more  specific  goals  and  objectives  provide  definition 
of  generalized  entities  which  must  be  provided  to  fuliiii  the  mission. 

The  actual  services  or  products  to  be  produced  by  the  library  are  determined 
fro®  the  goals  and  objectives  statements.  The  operations  statements  define 
the  actions  the  library  staff  performs  to  produce  the  services  or  products. 

In  applying  this  method  the  mission  statement  must  be  provided  first. 

It  can,  for  example,  be  prepared  by  the  director  of  the  research  laboratory. 

It  is  desirable,  however,  for  the  librarian  to  review  the  statement  to  in¬ 
sure  it  is  in  terms  he  can  understand.  The  goals  and  objectives  statement, 
however,  should  be  prepared  jointly  by  the  highe*-  authority  and  the  librarian. 
Identification  of  the  particular  services  and  products  should  be  performed 
by  the  librarian  who  by  training  and  experience  is  best  fitted  to  determine 
which  services  and  products  of  the  library  fill  the  specifications  of  the 
goals  and  objectives.  When  these  3  levels  of  statements  have  been  decided 
upon,  ths.  lior&rlan  is  ready  to  test  this  particular  approach. 

The  librarian,  fro®  the  background  of  his  knowledge  of  the  overall 
situation,  must  then  assign  weights  or  values  to  the  various  services  and 
products  his  organisation  produces.  These  ■'an  be  arbitrary  numbers  which 
have  little  or  no  meaning  per  se,  but  which  by  comparison,  each  with  the 
others,  give  the  relative  Importance  of  the  services  and  products  as  they 
support  the  goals  and  objectives.  For  purposes  of  simplicity  in  this 
exercise,  the  total  sum  of  these  arbitrary  values  is  equated  to  the  total 
"utils"*  of  the  library,  as  will  be  discussed  later.  The  librarian  would 
then  also  assign  values  for  each  operation  according  to  its  effect  in  adding 
value  to  service'  or  products. 

Type  5  data  were  sought  fro®  some  of  the  libraries  visited  during  the 
Phase  II  data  gathering  survey.  This  data  type  is  expected  to  reflect 
the  relative  values  of  operational  outputs*  toward  providing  given  services 
and  the  relative  values  of  services  in  suppoitlng  the  mission  of  the 
libraries.  As  discussed  in  the  Phase  I  report,  a  matrix  was  developed  to 
facilitate  a  test  of  the  l^br-rians1  ability  to  assign  relative  values  for 
operations  and  services  (see  pp.  6-10,  Phase  I  Report). 

Some  of  the  librarians  asked  to  provide  these  data  were  not  responsive. 
This  indicates  that:  (1)  the  values  sought  were  not  clearly  communicated 
or  understood,  (2)  the  librarians  are  often  not  able  to  quantify  the  values 
of  operations  ir.  supporting  services**  or  values  of  services  in  supporting 
missions. 

From  the  data  submitted  it  was  apparent  that  all  librarians  do  not 
assign  the  same  values  to  given  operations  and  services.  However,  there 
was  some  general  agreement  in  the  rank  order  of  values  assigned.  This  rank 

*  Utils,  an  arbitrary  unit  which  represents  the  amount  of  utility  or  value. 

**  In  this  discussion  the  term  'services"  includes  "products." 
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order  is  from  highest  to  lowest  of  5  services:  reference  search,  circula¬ 
tion,  bibliographic,  publications,  translations.  The  data  indicated  that 
even  though  libraries  have  different  missions,  goals  and  objectives,  the 
relative  values  of  operations  and  services  do  not  diverge  completely, 
although  values  can  be  expected  to  be  affected  by  missions  and  the  kinds 
and  degrees  of  operations  performed. 

In  evaluation  of  the  re*-  onses  received  during  the  data  collection 
effort  it  was  deduced  that  some  librarians  do  have  the  ability  to  assess 
the  comparative  values  of  their  services  and  operations.  However,  it  is 
considered  necessary  to  develop  the  concept  further  in  order  that  more 
meaningful  and  complete  value  data  may  be  collected  to  determine  and  validate 
criteria  from  Type  4  data.  With  this  objective  in  mind,  the  test  described 
in  the  following  section  will  be  conducted  in  Phase  III  of  this  contract 
to  collect  additional  and  more  com; rehensive  Type  5  data. 

Test  procedure  for  developing  and  validating  criteria  by  utility  analysis 
(see  Utility  Analysis  in  the  Glossary,  p.  97) 

The  procedures  outlined  here  investigate  the  applicability  of  utility 
criteria  for  measuring  the  value  of  services  and  products,  and  for  making 
determinations  of  equilibrium  conditions  which  maximize  utility  within 
given  budget  constraints.  The  application  of  utility  analysis  used  here 
is  net  based  upon  the  users  judgments  of  utility  as  in  the  classical  ap¬ 
proach.  The  measurements  of  utility  applied  here  are  based  upon  the  Judg¬ 
ments  of  a  librarian  as  to  the  utility  of  the  sendees  and  operations  in 
meeting  the  mission  of  his  library.  For  each  library  tested,  it  will  be 
necessary  to  have  statements  prepared  by  the  librarian  and/or  the  next 
higher  echelon  which  define: 

(1)  The  library  mission. 

(2)  The  library  goals  and  objectives  supporting  the  mission. 

(3)  The  particular  services  and  products  of  the  library  which 
Implement  the  goals  and  objectives. 

In  the  course  of  the  tests,  i  number  of  determinations  must  be  made. 
These  will  be: 

(1)  Determination  of  Inputs  and  outputs  of  each  service  and  the 
relationships  between  inputs  and  outputs. 

(2)  Determination  of  the  ictors  to  be  maximizid  for  each 
service  relative  to  the  purpose  of  the  library.  For  example, 
moat  private  industries  have  a  central  purpose  of  maximizing 
profits.  A  library  may  also  have  a  central  purpose  which  may 
be  to  maximize  the  percentage  of  vlthin-mlssion  needs  met  or 
to  maximize  tie  value  of  library  services  and  products  in 
supporting  research  and  development  activities  of  the  users. 

(3)  Determination  of  a  means  of  measuring  the  utility  of  the 
factors  to  be  raxlmlzed. 
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(4)  Determination  of  the  operations  to  be  performed  in  the  effort 
to  maximize  utility  of  each  service  or  product. 

(5)  Determination  of  standard  output*  units  per  man-hour  for  each 
operation . 

(6)  Determination  of  the  change  in  utility  (A  U  )  Of  each  service 

( i )  with  respect  to  the  change  in  each  operational  output  unit 
(aOj)  t  where  aU,/AOj  ■  marginal  utility  of  service  (I)  with 
respect  to  operation  (j)  .  Vo  determine  the  marginal  utility  of 

a  given  operational  output,  it  is  necessary  to  describe  a  utility 
curve  such  as  shown  in  Figure  19.  This  curve  is  described  by 
plotting  points  on  utility-output  coordinates.  Various  operational 
output  quantities  are  plotted  against  their  utility.  The  out¬ 
put  quantities  may  be  arbitrarily  chosen  to  deviate  from  existing 
outputs.  In  Table  9  the  existing  o  pu^  is  80  standard  man-hours 
of  work.  The  deviationt  are  in  10  man-hour  increments  above  and 
below  the  existin;  output.  Judgments  are  then  made  of  the  utility 
o£  each  service  derived  from  the  various  outputs.  The  marginal 
utility  of  the  existing  output  can  be  represented  by  the  average 
slope  of  the  utility  curve  between  points  3  and  4  ( P 3  &  P4)  or 
A  U 3 /A0j3  -  3C/10  -  3.0,  (or  ideally  at  P3) . 

(7)  Determination  of  the  total  change  of  utility  AT  Uj  of  operation 

(j)  with  respect  to  the  change  in  operational  output  A Oj where 
ATUj/AOj  -  MUj  »  marginal  utility  of  respective  operations 
outputs. 

(8)  Determination  of  th*  cost  per  mar.-hour  at  1001  utilization  for 
each  operation. 

If  these  determinations  can  be  made,  it  will  be  possible  to  establish 
criteria  '  r  allocation  of  man-hours  in  such  a  way  that  ove-all  utility  of 
operations  and  services  may  be  maximized.  The  equilibrium  condition  for 
maximum  utility  exists  when  total  budge*  for  operations  (I);  amount  of  each 
operational  output  (A,B,C, . . .N) ;  price  of  each  operational  unit  (Pa,  Pb » 
Pc,...Pn)  k.id  marginal  utility  of  each  operational  unit  MUa,  NUb ,  MUc,.,, 

MUn  satisfy  the  following  equations: 

(1)  I  «  APa  +  BPb  +  CPc  +...+NPn; 

(2)  MUa  -  MUb  -  MUc  -  ...  -  MUn. 

Pa  Pb  Pc  Pn 


*  The  tern  "output"  he*  different  meanings,  depending  on  the  part  of  the 
system  .0  which  it  refer*.  In  this  chapter,  unless  otherwise  stated, 
the  term  meant  a  measure  of  cperatlonal  actions,  not  a  measure  of  number 
or  quality  of  services  or  products.  In  this  sense  operational  outputs 
are  inputs  toward  producing  services  or  products. 
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I*,  these  equations  cannot  be  satisfied,  it  mav  be  possible  to 
optimize  manpower  allocation  to  approach  utilitv  maximization.  If  there 
are  constraints  other  than  budget  coistraints,  such  as  manpower  constraints, 
it  uay  not  be  possible  to  maximize  ■  tilitv.  Regardless  of  constraints  and 
barriers  to  utility  maximization  at  the  point  of  equilibrium,  criteria  may 
be  developed  to  Indicate  the  optimum  balance  between  budget  allocation  and 
utility . 

The  following  example  will  serve  to  demonstrate  the  utility  analysis 
approach . 

Example : 

Figure  20  is  a  hypothetical  total  and  marginal  utility  schedule  for 
estimates  of  long  run  utility  of  operations  and  services.  Only  8  services 
and  11  operations  are  considered  as  the  significant  operation  .  and  services 
at  a  hypothetical  library.  The  value  to  be  maximized  Is  the  utility  of 
the  services  in  meeting  the  wlthin-miasion  needs  of  the  users.  The  outputs 
units  being  measured  as  contriouting  to  utility  are  standard  work  units 
expressed  in  teraia  of  standard  man-hours  of  work  (Figure  20,  third  column 
from  right). 

The  numbers  underlined  in  the  schedule  are  existing  output  (standard 
man-hours  of  work)  and  utilitv  (utils)  measures  for  each  operation.  The 
standard  >"an-hours  of  work  produced  (in  the  output  column)  is  the  same 
number  *a  the  man-houra  worked  (in  the  operation  input  column),  wher  the 
operator  la  working  at  1003  utilization.  This  concept  and  the  criteria  for 
standards  will  be  discussed  further  in  the  Phase  III  development  of  the 
Croup  Attainment  Program.  For  cur  present  example  we  will  assume  that  all 
operators  are  working  at  i003  utilization.  The  other  numbers  in  the 
schedule  represent  utility  and  marginal  utility  changes  at  various  output 
changes . 

If  the  library  is  operating  at  a  point  of  maximum  utility  within 
bu^et  constraints,  the  following  conditions  must  be  satisfied: 
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■  Utilizing  the  output  data  In  the  3rd  column  from  the  right,  when  the 
total  budget  for  operations  Is  $2200/week,  condition  (1)  requires  that: 

■j 

$2200.00  -  (80)  ($3.00)  +  (A0)  ($5.00)  +  (30)  ($4.00) 

+  (100)  ($3.00)  +  (100)  ($3.00)  +  (20)  ($2.00) 

+  (20)  ($5.00)  +  (20)  ($4.00) 

+  (30)  ($4.00)  +  (30)  ($2.50)  +  (100)  ($6.00)- 

This  reduces  to: 

$2200.00  -  $2175.00. 

It  can  be  seen  chat  the  library  does  closely  approximate  the  first 
condition. 

Utilizing  the  marginal  utility  data  in  the  1st  coltsan  from  the  right 
and  the  prices  of  respective  output  units  condition  (2)  requires  that: 

8.3  -  1.0  -  .5  -  .4  -  .3  -  2.0 

3.00  5.00  4.00  3.00  3.00  2.00 

-  1.2  ■  7.0  -  1.5  -  1.1  -  9.3 
5.00  4.00  4.00  2.50  6.00  * 

This  reduces  to: 

2.80  -  .20  -  .13  -  .13  -  .10  -  1.00  -  .24  -  1.75  -  .38  -  .44  -  1.55, 


It  can  be  seen  that  the  library  does  not  closely  approximate  the 
second  condition.  Therefore,  in  order  to  maximize  utility  It  will  be 
necessary  to  add  output  to  operations  where  MUj/Pj-  the  largest  positive  number. 

Since  "^pg1  -  2.80  and  2.80  Is  the  largest  MUj/Pj,  we  should  add  output  (A) 
to  cataloging,  classifying  and  indexing  and  reduce  output  from  operations 
where  MUj/Pj  ■  the  smallest  number.  Since  -  the  smallest  number  »  .10, 

Pe 

we  shall  remove  10  units  of  E  (10  man-hours)  and  allocate  the  cost  of 
10  man-hours  of  E  (10  x  $3.00  -  $30.00)  to  A.  If  we  add  $30.00  to  A  we  will 
add  10  man-hours  to  A,  since  the  Pa  -  $ 3. 00 /man-hour . 

The  change  in  allocation  of  funds  causes  a  change  in  total  utility. 

The  sum  of  the  numbers  In  the  total  utility  column  which  are  underlined 
represent  the  total  utility  of  operations  before  the  change.  The  siza  of 
the  numbers  in  the  total  utility  column  which  are  In  parenthesis  represent 
the  total  utility  after  the  change.  Because  of  the  reallocatior  of  f>mds 
the  utility  of  E  drops  5  utils,  and  the  utility  of  A  increases  ly  83  utils. 
Therefore,  a  net  long  run  gain  of  78  utils  is  anticipated  by  spending  $30.00 
less  on  predetermined  circulstion  and  $30.00  more  on  cataloging,  classifying 
and  indexing. 
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At  this  point,  before  the  change  is  made,  the  purpose  for  the 
change  and  the  effect  of  the  change  must  be  revieved.  The  purpose  is 
to  maximise  utility  of  total  operations  in  meeting  the  vi thin-mission 

needs  of  users. 

The  effect  would  be  decreased  predetermined  circulation  and  increased 
cataloging,  classifying  and  indexing.  If  there  are  no  obvious  constraints 
upon  reducing  predetermined  circulation  or  increasing  cataloging,  classify¬ 
ing  and  indexing,  the  change  should  be  recommended,  reviewed  by  top 
management  and  implemented  if  approved.  Disapproval  would  constitute  a 
constraint  and,  therefore,  may  be  a  barrier  to  max  mixing  utility  of  the 
operations  in  meeting  the  within-mission  needs  of  the  users.  On  the  other 
hand,  disapproval  may  indicate  poor  value  judgments  in  the  analysis  which 
should  be  revised  to  improve  the  measure  of  utility.  If  there  are  barriers 
such  as  (1)  no  additional  qualified  eatalog-rs  are  available  or  (2)  top 
executives  insist  that  predetermined  circulation  should  not  be  reduced 
regardless  of  a  Jower  predicted  benefit  per  dollar  invested,  then  It  will 
be  necessary  to  optimise  service  utility  'v  seeking  otl.ar  changes  which  will 
resolve  greater  total  utility.  Perhaps  adding  man-hours  to  H  and  reducing 
man-hours  of  C  01  D  may  be  accomplished. 

Operation  D  is  circulation  actions.  Assuming  that  the  personnel  per¬ 
forming  circulation  actions  are  fully  utilized  in  serving  clients,  the  man- 
hours  reduced  must  be  accomplished  by  improving  methods  of  circulation 
operations,  otherwise  reduction  in  man-hours  could  cause  a  bottleneck  which 
reduces  user  satisfaction. 

The  utility  approach,  therefore,  is  useful  in  pointing  out  the  opera¬ 
tions  whit.*  chould  be  concentrated  upon  first  to  improve  methods  and 
ciency  in  order  to  free  labor  for  more  effective  and  utility  maximizing 
oparations . 

In  the  case  of  increased  total  manpower  for  operations  the  additional 
man-hours  should  be  added  where  MU]/ is  largest.  In  ’•'ne  case  of  staff 
cuts  the  man- hours  should  De  reduced  where MUj/H;  is  smallest. 


APPENDIX  A 


GLOSSARY  OF  MANAGEMENT  SCIENCE  TECHNIQUES 


Although  a  glossary  of  management  science  terms  was  presented  as 
Appendix  E  of  the  Phase  I  Report,  a  revision  of  several  of  the  glosses 
is  required.  This  revised  version,  therefore,  provides  Modified  ex¬ 
planations  of  several  of  the  management  techniques  of  the  original  glossary . 
However,  it  does  not  present  or  define  terms  which  do  not  represent  formal¬ 
ized  management  techniques  par  ee,  such  as  the  word  "budget." 

The  purpose  here  is  to  reorganize  the  major  techniques  considered  in 
Phase  I,  to  add  some  new  considerations  and  to  provide  reference  numbers 
to  selected  articles  which  discuss  the  techniques.  Reference  numbers  through 
758  represent  either  abstracts  or  simply  references  to  articles  as  shown 
in  the  Phase  I  Report.  Numbers  above  758  represent  references  included 
in  Appendix  B  to  this  Phase  II  Report. 

The  references  refer  to  publications  which  relate  to  the  techniques 
in  one  or  more  of  the  following  respects: 

(1)  Methodology. 

(2)  Possible  application  to  industry,  government  and/or  libraries. 

(3)  Previous  application  in  industry,  government  end/or  libraries, 

(4)  General  discussion — edvantages,  disadvantages,  philosophy,  3tc. 

Following  the  references  are  parameters  explaining  what  each  technique 
indicates  or  controls. 

The  statements  and  parameters  are  not  intended  to  be  criteria  for 
measurement  but  presented  here  as  hypothetical  tools  for  measurement* 
These  hypothetical  fools  which  have  been  selected  as  tentative  tools  (see 
paired  comparison  experiment  page 60)  are  identified  in  the  tentative 
tools  chercs,  Figures  10  and  11.  In  these  charts  each  tool  (technique) 
is  identified  with  the  library  operation  or  service  which  it  measures  or 
controls. 

Each  of  these  tentative  tools  is  potentially  useful  for  developing 
criteria  for  measuring  or  controlling  the  efficiency  and  effectiveness  of 
library  services  and  operations.  In  light  of  the  data  gathered  and  the 
observations  made  during  the  Phase  II  library  survey,  certain  of  these 
tentative  tool.*  will  be  selected  to  develop  criteria  in  Phase  III  of  this 
contract . 

I .  Systems  Analysis 

Systems  Analysis  is  a  functional  process  responsible  for  the  develop¬ 
ment  of  policies  and  objectives  for  the  planning,  acquisition  and 
utilisations  of  library  resources  and  requirements  as  affected  by 
the  operators,  users  and  designers  of  library  information  systems 


at  various  levels  tad  mission  assignments  considering  political, 
social  and  economic  factors.  This  basic  approach  could  be  useful 
to  segregate  and  delineate  the  individual  functions  of  the  library. 
Systems  analysis  will  show  the  interfaces  between  functions  and 
their  relationship  to  the  overall  objectives  and  goals  of  the 
library.  This  concept  has  been  expanded  to  include  the  relationship 
of  the  library  to  the  users  and  to  the  parent  organization  and  the 
mission  of  that  organization.  In  this  sense  the  system  is  'a  set 
of  organized  operations  to  satisfy  a  definable  user  requirement." 
(References:  38,  46,  49,  126,  155,  157,  146,  282,  327,  410,  416.) 

Indicates  efficiency  of  individual  operations  of  the  library  by 
determination  of  the  ability  of  the  various  operations  to  work 
together.  The  total  efficiency  of  sn  organization  or  a  system  is 
a  function  cf  the  efficiencies  of  each  operational  unit  and  the 
efficiency  of  the  interactions  between  the  operational  units.  Systems 
analysis  can  be  used  to  facilitate  measuring  the  efficiency  of  and 
between  operational  units  by  models,  charts,  etc.,  which  simulate  an 
on-going  system.  This  simulation  may  be  used  in  cost-eff ectivenes* 
analysis,  P.P.E.S.  or  other  systems  techniques  which  indicate,  measure 
or  predict  parameters  of  a  system. 

Indicates  effectiveness  by  identifying  the  individual  functional 
units  of  the  library  and  delineating  the  roles  and  goals  of  each  with 
regard  to  the  overall  mission  of  the  library.  After  this  role  and 
goal  assigning  has  been  accomplished,  simulations  which  identify 
probabilistic  and  deterministic  variables  can  be  made.  The  simulations 
facilitate  judgments  as  to  each  unit's  effectiveness  in  attaining 
goals.  Even  though  all  functional  units  are  effective  in  their  roles, 
organizational  effectiveness  is  not  insured.  The  outputs  of  each  unit 
must  merge  into  the  system  in  an  effective  manner.  Systems  analysis 
should  also  determine  the  effectiveness  of  the  coordination  between 
goal  directed  roles  and  user  needs  through  user  feedback. 


II .  Coat-Effectiveness  Analysis 

This  is  an  economic  analysis  applied  to  resource  allocation.  It 
relates  tangible  costs  to  meaningful  and  utility-maximizing  missions 
or  programs,  instead  of  to  the  cost  of  individual  segments.  In 
economic  analysis,  the  concern  is  with  objectives  rather  than  objects. 
Cost-effectiveness  can  be  determined  by  identifying  the  performance 
or  accomplishments  of  a  program  needed  to  achieve  objectives,  and 
evaluating  all  resources  associated  with  the  program  in  relation 
to  the  benefits  anticipated  from  the  program.  (References:  14,  20, 

53,  86,  112,  115,  125,  130,  141,  155,  232.  260,  303,  429.) 
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Indicates  ef f Icency-e f fectlveness  (Performance)  level  by  correlating 
efficiency  (expressed  in  terms  of  coet)  and  effectiveness  (expressed 
in  terms  of  probabilities  of  meeting  objectives  or  in  terms  of 
qualitative  judgments  of  effectiveness).  Cost-effectiveness  analysis 
helps  resolve  the  optimum  cost-effective  system  and  thereby  helps 
to  maximize  performance.  Optimum  cost-effectiveness  can  be  indicated 
after  assigning  weights  to  cost  by  -ost  accounting  (sie  Coet  Accounting, 

III)  and  effectiveness  by  qualitative  or  quantitative  measures.  Graphs, 
computer  programs  (Linear  Programming,  VII)  or  models  (Models,  XIX) 
can  be  used  to  indicate  the  relative  cost-effective  level  of  alternate 
methods  or  systems. 

III.  Cost  Accounting 

Cost  accounting  is  an  excellent  control  device,  commonly  used  in 
industry,  which  allocates  the  various  costs  of  doing  business  to  the 
units  produced  or  sold.  Cost  accounting  serves  a  need  for  communication 
and  control  between  sources  of  money  and  management  concerning  the  use 
of  capital  invested.  As  used  in  the  library,  cont  accounting  could 
develop  a  ,_ost  figure  for  each  operation  performed  and  each  product  or 
service  given  by  utilizing  cost  data  derived  from  accounting  records, 
standard  data  (Standard  Data,  XIII)  or  tailored  techniques  such  as 
the  systems  generalized  models  which  have  been  developed  in  this  study 
for  each  library  service.  (Reference:  62,  63,  111,  226,  231,  311,  367, 

760.) 

Indicates  efficiency  by  identifying  costs  of  each  service  or  operation 
in  terras  of  dollars.  Efficiency  of  a  service  or  operation  can  be 
measured  in  terms  of  dollars  when  the  outputs  are  constant  in  terms 
of  quality  or  effectiveness.  If  outputs  vary,  the  cr  j  must  be  pro¬ 
rated  to  indicate  efficiency. 

Controls  Performance  by  periodic  checking  of  expenditures  with  budget 
allocations  and  automatic  feedback  control  when  costs  become  inconsistent 
with  budgets  (Budgeting,  XXXI). 

IV .  Utility  Analysis 

flinty  analysis  depends  upon  the  concept  that  the  consumer  not  only 

tows  that  he  prefers  A  to  B,  but  he  cm>  give  numerical  expression  to 
his  desires  which  permits  him  to  say  he  prefers  A,  say,  wice  as  much 
as  B,  giving  A  twice  the  utility  of  B.  (References:  102,  183,  184,  303, 

361,  362,  372,  763.) 

Indicates  effectiveness  by  evaluating  a  product  or  service  in  terms  of 
how  much  one  is  willing  to  sacrifice  in  order  to  get  that  product  or 
service.  This  sacrifice  nay  be  manifested  as  a  tradr  'f f  of  money,  time, 
effort  or  other  products  or  services.  The  cardinal  utility  of  the  desired 
product  or  service  can  be  measured  by  dollars  or  time  units  and  by  other  cow- 
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parativ*  units  such  as  weights  of  the  telativt  preference  for  one 
product  or  service  over  another.  Care  must  be  taken  not  to  assuae 
utility  can  be  measured  by  any  absolute  value  of  what  o ue  is  willing 
to  sacrifice.  Ordinal  utility  of  a  commodity  for  one  Individual  can 
be  ■ensured  by  his  relative  indifference  between  having  various 
cooaodlties.  If  a  population  of  scientists  has  the  same  indifference 
between  having  Chemical  Abstracts  or  Current  Contents,  the  utility  of 
the  two  is  constant  for  the  group.  However,  if  40%  of  the  scientists 
prefer  one  and  60%  prefer  the  other,  the  utility  of  each  depends  on 
the  user  indifference.  Utility  of  various  items  in  a  collection  can 
b*  rated  by  comparison  wl^h  each  other.  For  instance,  rating  or  rank¬ 
ing  techniques,  su^h  as  paired  comparisons  (Figure  9),  can  be  made 
between  a  number  of  journals  to  determine  the  utility  rank  of  each 
journal  for  each  scientist.  The  ratings  or  ranks  given  to  each  journal 
can  be  averaged  to  give  a  mean  utility  rank  for  each. 

V .  P.P.B.S.:  Planning-Programming-Budgeting  System 

P.P.B.S.  is  a  set  of  procedures  receiving  increasing  use  and  importance 
in  the  preparation  of  government  agency  budgets,  which  specify  program 
objectives  in  quantitative  terms,  measure  benefits  and  seek  least  cost 
solutions  of  meeting  objectives  and  which  impose  controls  by  a  budgeting 
process.  The  Planning-Programming-Budgeting  System  requires  management 
to  think  and  plan  further  into  the  future,  to  delineate  objectives, 
analyse  costs  and  benefits  of  existing  programs  (costs  of  activities, 
equipment,  etc.?  benefits  from  probabilities  of  accomplishing  goal- 
direoted  events  and  objectives) ;  and  find  better  and  cheaper  ways  to 
accomplish  objectives.  P.P.B.S.  is  founded  on  3  major  concepts: 

1.  Each  agency  possesses  in-house  analytical  capability  for 
determining  the  objectives  and  programs  which  they  should 
support . 

2.  A  multiyear  planning  and  programming  process  incorporating 
and  using  a  systems  approach  which  classifies  and  presents 
essential  data  required  for  decision-making  (such  essential 
data  can  be  utilized  in  Mathematical  &  Econometric  Models, 

XIX,  or  Systems  Models). 

3.  A  budgeting  process  which  can  refine  broad  program  decisions 
for  subsequent  review  and  action  (Budgeting,  XXXI). 

(References:  437,  317.) 

Controls  performance  by  budgeting.  Budget  allocations  are  determined 
by:  (1)  planning,  (2)  programming  essential  data  about  resources 

required  for  decision-making  and  (3)  processing  these  data  to  develop 
base  lines  of  resource  limits  as  criteria  for  budgeting.  This  tech¬ 
nique  was  primarily  designed  for  long-run  control  by  budgeting,  re¬ 
quiring  periodic  review  and  analysis  of  programs  and  budgets,  and  v 

revising  resource  allocation  where  necessary.  Jf 

4 

1 


i 
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VI .  Statistical  Sampling 


A  method  of  selecting  a  sample  froa  a  population  to  Insure  its 
being  random  and  consequently  representative  of  the  entire 
population.  The  saaple  reflects  inferences  about  the  parameters 
of  the  whole  population  according  to  statistical  theories. 

Statistics  is  a  general  tern  for  a  large  group  of  aatheaatlcal 
tools  baaed  on  laws  of  probability,  used  to  collect,  analyse  and 
Interpret  numerical  data.  The  purpose  of  statistical  analysis  is 
to  provide  methods  of  treating  data  so  that  the  a ax in urn  information 
can  be  obtained  with  a  predetermined  risk  of  drawing  false  conclusions. 
No  method  of  analysis  can  extract  more  information  from  a  set  of  data 
than  is  contained  therein,  and  no  method,  statistical  or  otherwise, 
can  draw  conclusions  from  experimental  data  with  zero  risk  of  error. 
(References:  4,  386,  759.) 

Indicates  efficiency  and  effectiveness  by  presupposing  that  information 
about  effectiveness,  efficiency  or  performance  can  be  extracted  from 
critical  data.  It  further  assumes  that  it  is  no*,  necessary  to  study 
all  data,  but  it  la  possible  to  examine  a  representative  sample  of 
the  data  to  make  Inferences  about  efficiency,  effectiveness,  perform¬ 
ance  or  any  other  parameter  of  the  population  reflected  by  the  sample. 

VII.  Linear  Programming 

Any  problem  concerned  with  minimizing  or  maximizing  linear  objective 
functions,  such  as  total  costs  or  net  profit,  and  subject  to  a  set 
of  linear  equations  or  inequalities  in  the  form  of  constraints  due  to 
limitations  of  men,  materials,  capital  or  other  resources,  is  a  linear 
programming  problem.  The  two  most  common  types  of  linear  programming 
are  the  graphic  and  simplex  methods.  The  analytical  procedure  in  all 
Instances  consists  of  4  parts: 

1.  Arranging  the  alternative  possible  goals  to  be  sought. 

2.  Defining  the  assumptions  to  be  employed. 

3.  Determining  the  balance  of  net  advantages  and  disadvantages 
in  selecting  the  optimum  solution. 

4.  Modifying  selection  by  relating  the  alternatives  to  the 
organization's  overall  objectives. 

(References:  220,  538;  also  pp.  53  and  54,  Phase  I  Report.) 

Indicates  efficiency  and  effectiveness  by  resolving  optimum  efficiency  - 
effectiveness  from  data  containing  many  Interrelated  factors  which  can 
be  expressed  in  linear  equations  or  inequalities.  Actual  efficiency, 
effectiveness  and/or  performance  values  can  be  compared  to  the  optimum 
values  to  indicate  the  differential  between  actual  and  optimum 
"attainable"  accomplishments. 
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VIII.  Correlation  Analysis 


The  coefficient  of  correlation  provides  a  measure  of  the  degree 
of  relationship  between  variables.  Linear  correlations  have  a 
possible  range  of  -1.00  to  +1.00.  Curvilinear  correlations  have 
a  range  of  0  to  +1.00.  If  one  variable  increases  while  the  other 
decreases,  the  result  will  be  a  negative  coefficient  in  linear 
correlations.  If  there  is  no  relationship  between  the  variables, 
the  coefficient  will  be  zero.  In  general,  the  closer  the  coef¬ 
ficient  of  correlation  comes  to  enualing  +  1,  the  better  the 
relationship  is  for  forecasting  purposes.  Correlation  analysis 
has  been  extended  in  this  definition  to  include  regression  analysis — 
a  method  for  computing  lines  of  best  fit  or  formulas  showing  re¬ 
lationships  between  variables.  (References:  4,  16,  23,  212,  310, 

768.) 

Indicates  factors  contributing  to  efficiency  and  effectiveness  by 
disclosing  relationships  between  variable  factors  and  efficiency, 
effectiveness  or  performance.  For  example,  if  high  correlation 
exists  between  effectiveness  and  the  number  of  books  in  a  library 
collection,  it  can  be  assumed  that  effectiveness  ia  a  function  of 
the  extent  of  the  collection.  Furthermore,  if  linear  relation¬ 
ships  exist  between  effectiveness  and  extent  of  collection  within 
a  range,  effectiveness  can  be  predicted  by  the  number  of  volumes 
within  that  range.  However,  the  number  of  volumes  must  be  taken 
as  only  one  component  of  a  measure  of  total  effectiveness.  If  it 
were  possible  to  establish  all  variables  which  determine  effective¬ 
ness  and  all  correlations  between  these  variables,  total  “ffective- 
neas  could  be  predicted  by  weighting  the  correlation  values,  of 
the  variables,  according  to  their  power  to  contribute,  and  summing 
the  weighted  values. 

IX .  Organization  Chart 

Graphically  Identifies  bureaucratic  relationships  between  functional 
a..u  staff  units  within  an  organization  and  delineates  areas  of 
responsibility  of  each  unit.  (References:  79,  337.) 

Indicates  efficiency  by  providing  a  view  of  the  distribution  of 
responsibilities  making  recognition  of  role  inconsistencies  with 
organizational  goals  easier. 

Controls  performance  by  providing  a  means  to  communicate  proper 
relationships  between  and  proper  roles  of  the  line  and  staff  units. 

X.  Layout  Chart 

Usually  a  floor  plan  of  a  facility  locating  the  items  in  the  physical 
layout.  Three  types  of  layout  charts  are  plant  layout,  department 
lay  t  and  workplace  diagrams.  (Reference:  761.) 

) 
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XI.  Methods  Study 


XII. 


XIII. 


Method  study  is  a  systematic  anaiycis  of  work  to: 

1.  Eliminate  unnecessary  work. 

2.  Arrange  the  remaining  work  in  the  best  possible  order. 

3.  Standardize  usage  of  proper  work  methods. 

4.  Establish  accurate  time  standards  for  the  work. 

(References:  32,  255,  315,  761.) 

Indicates  efficiency  relative  to  standardized  methods  and  times 
for  elements  of  work  such  as  basic  motions  or  operations.  These 
st&idards  are  developed  by  coordination  of  motion  studies  and  time 
studies.  The  standardized  methods  and  times  are  compiled  and 
presented  in  various  forms  such  as  general  standard  data  (basic 
motion  times)  and  specific  standard  data  (basic  operati.ua  times) 
(Standard  Data,  XIII).  Standard  data  are  time  values  and  the  end 
product  of  motion  and  time  studies. 

Standardization 

An  authority  or  rule  for  the  measure  of  quantity,  extent,  value  or 
quality,  denoting  the  establishment  of  practices,  policies  and  rules. 
Also  connotes  simplification  (reduction  of  diversification)  and 
denotes  establishment  of  standard  equipment  and  standard  methods. 
(References:  11,  51,  58,  74,  81,  83,  94,  118,  135,  170,  171,  183, 

185,  196,  224,  233,  242,  269,  296,  382,  355,  400,  481.) 

Controls  performance  by  establishment  of  vractices,  policies, 
rules,  standard  equipment  and  standard  methods. 

Standard  Data 


Standard  data  are  predetermined  time  values,  tabularized  or  reduced 
to  simplest  terms,  and  compiled  to  largest  time  values  consistent 
with  accuracy,  flexibility  and  speed  required  for  the  establishment 
of  time  standards  for  operations.  The  work  factor  system  divides 
standard  data  into  2  classes,  general  and  specific.  General  standard 
data  are  time  values  developed  for  a  relatively  small  work  segment, 
such  as  basic  motions  and  usually  ap7licable  to  several 
classes  of  work.  Specific  standard  data  are  usually  developed  for 
larger  work  segments  occurring  in  specific  operations  or  classes  of 
operations.  The  time  values  in  specific  standard  data  are  often 
developed  from  combinations  of  values  selected  from  a  general  system. 
(References:  32,  41,  315,  449,  761) 

Indicates  efficiency  by  relative  comparison  of  actual  time  to  standard 
time  to  indicate  the  index  of  productivity  or  percent  utilization. 

Controls  performance  by  setting  an  attainable,  expected  and  en¬ 
forceable  standard  of  efficiency. 
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XIV.  Performance  Evaluation 


Gathering  of  performance  information  (such  as  qualitative 
and  quantitative  measures  of  efficiency  and  effectiveness) 
and  appraisal  of  this  information  and  managerial  philosophy. 
(References:  80,  120,  132,  162,  166,  289,  324,  462.) 

Indicates  efficiency  and  effectiveness  by  subjective  appraisal 
of  qualifying  factors  of  behavior  and  abilities  of  individuals, 
supported  by  objective  appraisal  of  quantitative  statistics  such 
as  index  of  utilization,  job  description  analysis  data  (Job 
Description  Analysis,  XVIII)  and  degree  of  effectiveness  of  the 
Individuals  in  meeting  organizational  goals.  Appraisals  should 
be  made  by  trained  evaluators. 

XV .  Work  Sampling 

A  method  for  analyzing  work  to  find  delay  allowances  applicable 
to  a  given  job,  to  determine  percentages  of  machine  and  operator 
utilization  and  to  establish  production  standards  (Standard  Data, 
XIII  and  Standardization,  XII).  It  utilizes  random  observation  of 
operators  to  determine  ststistically  the  amount  of  time  spent  on 
each  activity  (work  or  non-work)  and  the  pace  at  which  the  operators 
work.  (References:  32,  315,  465,  587,  761.) 

Indicates  efficiency  by  determining  the  utilization  of  the  operators 
studied.  This  utilization  can  be  measured  in  terms  of  percent  time 
spent  working  at  100Z  pace.  Work  time  ia  defined  as  the  time  on  the 
job  minus  non-work  time,  avoidable  rework  time  and  unnecessary  work 
time.  During  work  time  the  operator  ia  tired  according  to  pace. 

The  operator's  percent  utilization  will  be  his  work  time  multiplied 
by  his  average  pace  rating  divided  by  the  total  time  on  the  job. 


NOTE:  All  work  sampling  techniques  do  not  require  pace  measurements 
however,  utilization  percentage  figured  without  pace  consideration 
cannot  be  expected  to  be  as  good  a  measure  of  efficiency  as  utiliza¬ 
tion  percentage  including  pace  leveling. 

XVI .  Group  Attainment  Program  (GAP) 

using  proven  measurement  techniques  (such  as  Work  Sampling,  XIV  and 
Standard  Data,  XIII),  GAP  objectively  determines  attainment  factors 
for  the  jobs  in  a  group.  These  attainment  factors  represent  the 
time,  including  allowances,  that  it  should  take  a  reasonably  com¬ 
petent  employee  using  prescribed  methods,  and  working  at  a  normal 
pace,  to  perform  one  operation  successfully  and  fullv.  Having 
determined  the  "should  take"  time  for  all  operations,  it  's  applied 
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Co  the  present  or  planned  volume  of  operations  to  determine 
the  total  labor  requirements  for  the  entire  volume  of  services 
or  for  any  given  type  of  service.  Attainment  factors  or  "shoulo 
take"  times  are  recorded  as  specific  standard  data  (Standard  Data, 
XIII).  (Reference:  251.) 

Indicates  efficiency  of  groups  by  comparison  of  actual  time  spent 
to  '‘should  take'  time.  The  measure  of  efficiencv  is  expressed  in 
terms  of  an  index  of  productivity  which  is  computed  by  dividing 
"should  take"  time  by  actual  time. 

XVII.  Human  Relations 

The  study  of  the  kinds  and  degrees  of  interactions,  and  reactions 
between  people  and  their  orientation  to  groups  and  institutions 
and  the  reasons  for  such  interactions,  reactions  and  orientations. 
Studies  ere  based  on  the  social  sciences,  psychology  and  economics. 
Human  relations  is  primarily  an  analytical  technique;  however, 
public  relations  will  be  considered  an  application  of  human 
relations  as  an  operative  technique  which  applies  knowledge  of 
human  relations  to  control  interactions,  reactions  and  orientations. 
(References:  24,  47,  80,  108,  551.) 

Indicates  effectiveness  by  determination  of  how  well  staff  members 
are  suited  for  their  roles  and  how  well  clients  orient  to  library 
resources.  For  example:  Observations  before  and  after  clientele 
and  staff  training  or  public  relations  programs  can  indicate  how 
effective  the  programs  or  the  library  services  were  relative  to 
increased  interest  and  anility  to  utilize  library  resources. 

Controls  performance  by  creating  conditions  conducive  to  efficient 
and  effective  interaction  between  clients  and  staff.  Also  controls 
performance  by  determination  of  client  and  staff  needs  and  the 
means  to  meet  these.  For  example:  By  accommodating  staff  and 
clientele  needs  by  improving  library  facilities  and  working  conditions, 
by  reducing  boredom  and  fatigu«_,  and  bv  creating  interest  and  other 
juch  incentives  conducive  to  efficient  md  effective  work  in  and 
use  of  the  library. 

XVI 11 .  Job  Description  Analysis 

An  analysis  of  each  Job  in  terms  of  the  duties  of  the  job,  amount 
of  skill,  effort  and  responaibi li tv  required  and  *he  working 
conditions,  leading  u:  decisions  in  selection,  placement  and 
assignments,  training,  transfer,  upgrading  and  promotion,  and  in 
mak '  ,g  wage  surveys  and  adjustments.  (References:  264  ,  344,  428,  46?.) 


Indicates  efficiency  and  effectiveness  bv  identifying  the 
requirements  of  each  job  such  that  staff  abilities,  training, 
education,  experience  and  other  job  factors  can  be  weighted  and 
utilized  to  the  greatest  advantage.  This  advantage  is  realized 
by  optimise  usage  of  staff  qualifications  in  matching  people  with 
Jobs.  After  optimum  oatc'.,<ng  is  accoaspl ished ,  Jobs  with  under- 
qualified  workers  aav  be  ,'*u »  <zed,  indicating  a  need  for  training 
programs  to  increase  performance.  Jobs  with  ovcrq'  lfied  and 
high-salaried  workers  performing  routine  tasks  aav  also  be  rec¬ 
ognized,  indicating  a  need  to  assign  them  to  more  strategic 
operations,  Job  description  analysis  indicates  efficiency  of  al¬ 
location  and  utilization  of  staff  qualifications  and  wages 

XIX.  Models  -  Mathematical  and  Econometric 

Mathematical  -  In  Mathematical  analysis,  the  word  "model"  is 
used  to  mean  a  mathematical  description  of  an  operation  which 
represents  the  relationships  among  various  elements  with  sufficient 
accuracy  to  predict  the  actual  outcome  under  anv  expected  set  of 
circumstances.  T  e  advantage  of  a  model  is  that  it,  instead  of  the 
organization  it  simulates,  can  be  manipulated  in  a  variety  of  ways 
until  the  best  solution  is  found.  Tne  disadvantage  is  that  no 
model  can  duplicate  reality  completely. 

Econometric  -  A  "model"  based  on  the  idea  that  changes  in  economic 
activity  can  be  explained  bv  a  set  of  relationships  among  economic 
variables.  It  explains  the  past  and  predicts  future  economic 
activity  bv  mathematical  equations  that  express  the  most  profitable 
interrelationships  within  a  set  of  economic  variables.  Hie  best 
mathematical  arrangement  is  a  model  which  takes  the  form  of  an 
equation  or  system  of  equations  that  best  describes  the  pa»L  set 
of  relationships  according  to  economic  theorv  and  statistical  analysis. 
The  model,  in  othei  words,  is  a  simplified  abstraction  of  a  real 
situation  expressed  in  equation  for*  snd  applied  as  a  svstem  which 
will  yield  numerical  data  reflecting  the  Drobablc  outcomes  of  a  real 
economic  situation  when  certain  inputs  are  varied.  (References:  14, 

23,  46,  31,  55,  bk,  83,  23?,  410,  415,  76t  767.) 

Indicates  efficiency  and  effectiveness  bv  manipulating  variables 
within  the  model  to  resolve  the  efficiency  maxlrai ring ,  effective¬ 
ness  maximizing,  or  ef f iclencv-ef fectiveness  optimizing  conditions 
snd  bv  comparing  the  maximum  or  optimum  conditions  to  the  actual. 

XX .  Budgeting 

Developing  of  a  budget  bv  considering  organizational  needs  and  most 
efficient  and  effective  allocation  of  resources.  A  budget  is  a 
plan  for  authorization  of  resources,  which  can  be  expressed  in  terms 
of  people  or  aer.ev.  Generally  a  budget  converts  resources  into 
money.  Authorization  bv  means  of  the  budget  mav  be  made  to  a 


function  ,  *  division  or  the  total  organization.  The  budget 
should  be  correlated  to  planning  for  efficient  and  effective 
accomplishment  of  objectives.  It  should  represent  a  decision 
on  che  utilization  of  resources  after  alternative  courses  of 
action  are  evaluated.  The  budget  gives  management  a  quantitative 
base  for  measurement  and  evaluation.  It  represents  delegation 
and  delineation  of  responsibility  and  proaotea  organizational 
stability  and  continuity.  (References:  29,  63,  77,  111,  136,  226, 

231,  274,  338.) 

Controls  performance  by  allocating  resources  in  such  a  wav  as  to 
limit  waste  while  allowing  enough  resource  for  each  function 
to  maintain  optimum  cost-effectiveness  .  the  services  and 
operations . 

XXI.  PERT  (Program  Evaluation  and  Review  Technique) 

PERT  is  a  development  planning  and  control  concept  designed  to 
focus  managerial  attention  on  key  program  parts,  point  up 
potential  problem  areas  which  could  disrupt  program  goals,  evaluate 
progrer.  toward  the  attainment  of  program  objectives,  give  manage¬ 
ment  a  prompt  mechanical  reoor-ing  device,  and  facilitate  decision¬ 
making,  In  the  accomplishment  of  these  objectives,  PERT  uses  r*me 
units  as  a  common  denominator  to  reflect  3  categories  of  factors 
which  influence  success — progress,  resource  applications  and  re¬ 
quired  performance  specif icatiooa .  Some  of  the  important  t ^RT 
terms  are  given  below. 

Activity  -  An  activity  is  a  time-consiraing  phvsical  and/or  mental 
process.  It  is  the  work  in  process  between  2  points  or  events  in 
time.  An  activity  is  represented  bv  sn  arrow  in  PERT  network  svstema. 

Activity  time  -  Estimate  of  the  time  required  to  complete  an  activity 
in  a  specified  manner.  There  are  2  types  of  estimate’": 

1.  Single  estimates. 

2.  Three  time  estimates:  Expected  time  «*  a  +  4b  +  c  , 

where  ® 

a  •  Optimistic  time  *  the  shortest  time,  in 

which  an  activity  can  be  completed  if  there 
are  no  prob lews , 

b  »  most  .kely  time  “  the  estimated  time  to  complete 
che  activity  under  normal  working  conditions. 

This  is  the  model  estimate  of  time  that  would 
occur  most  often  if  the  activity  were  repeated 
under  exactly  the  same  conditions  aanv  times. 


c  *  pessimistic  time  ■  the  longest  time  an  activity 
would  take  if  major  changes  in  approach  or  design 
were  required;  for  example,  if  the  product  were 
more  difficult  than  it  first  appeared. 

Event  -  A  specific  accomplishment  recognizable  by  a  particular 
instar. £  in  time. 

Network  or  flow  plan  -  A  dia-'i-am  or  topological  representation  of  a 
project  made  up  of  one  or  more  series  of  sequential  events  joined  by 
activity  lines  to  show  the  time  and  relationships  among  events. 

Stack  ■  The  difference  between  the  latest  allow  ble  date  and  the 
expected  date  (T^  minus  Tj?)  .  Slack  may  be  positive,  zero  or  negative. 

Milestones  -  The  significant  events  or  selected  points  on  which 
progress  information  is  needed  in  order  to  evaluate  performance  with 
regard  to  meeting  final  project  objective. 

Critical  path  -  the  path  within  the  PERT  network  Cat  contains  the 
algebraically  least  amount  of  slack.  (References:  163,  294,  371, 

436,  4  )uf  4; 2.) 

Indicates  efficiency  by  providin''  view  of  key  program  development 
parts  and  their  labor  and  time  r  ireraents  with  bench  marks  to 
measure  progress. 

Controls  performance  by  providing  a  schedule  representing  ranges 
of  time  within  which  each  event  is  expected  to  be  satisfactorily 
completed  and  bv  providing  mechanisms  to  ''acilitate  decisionmaking 
when  milestone  ar''  «ot  falling  in  chronological  sequence  as  planned. 

XXI I ,  Data  Processing  Techniques  Evaluation 

Evaluation  of  methods  of  storing  and  retrieving  information  and 
performing  operation"  to  obtain  a  desired  output. 

Indicates  efficiency  of  data  processing  by  objective  analysis  of 
existing  systems.  Evaluation  of  data  processing  techniques  should 
be  conducted  by  persons  familiar  with  the  sciences  of  data  process¬ 
ing  and  familiar  with  various  systematic  approaches  to  evaluation 
of  data  processing  techniques. (1 ,  23,  46,  48,  49,  52,  54,  55,  56, 

85,  112,  160 ,  173,  181,  195,  200,  267,  376,  380,  415,  434.) 

Controls  performance  by  objective  evaluation  of  existing  systems, 
recognition  of  potential  areas  for  improvement  and  by  •'vgtematic 
development  of  proposals  for  change, (1,  46,  48,  49,  54,  56,  72,  144, 
160,  173,  181,  194,  195,  200,  201,  230,  267,  312,  376,  391,  425, 

434.) 
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XXIII.  Cybernetics 


Cybernetics  is  the  study  of  control  principles  applicable  to 
mechanical,  electrical,  biological,  organizational  and  economic 
systems  and  the  similarities  among  all  of  these  systems.  Efficient 
control  normally  works  with  a  low  expenditure  of  energy,  exerting 
just  enough  influence  to  effect  the  degree  or  kind  of  output 
desired.  (References:  518,  521,  755.) 

Controls  performance  by  the  study  of  an  implementation  of  sound 
control  principles.  Cybernetics  can  play  an  important  part  in 
increasing  efficiency  and  effectiveness  on  all  systems  levels  from 
coordination  of  inputs  to  quality  control  of  outputs.  Cybernetic 
principles  can  be  used  in  controlling  performance  of  automated 
systems.  For  our  purpose  in  this  phase  it  is  not  feasible  to 
propose  how  cybernetics  should  be  utilized  to  control  library 
automated  systems.  It  is,  however,  reasonable  to  hypothesise  that 
the  use  of  feedback  mechanisms  will  facilitate  control  to  the  extent 
that  automated  operations  can  monitor  themselves  or  provide  check 
points  for  management  follow-up  and  evaluation  of  performance. 

XXIV .  Quality  Control 

Control  of  quality  by  inspection  and  rejection,  repair,  correction 
or  acceptance  according  to  a  predetermined  standard.  (References: 
583,  588.) 

Indicates  effectiveness  by  statistical  computations  which  reflect 
the  effectiveness  of  a  volume  of  work  according  to  findings  made 
while  inspecting,  either  by  sample  inspection  or  100%  inspection 
(Statistical  Sampling,  VI)  of  that  work. 

Controls  performance  by  inspection  of  a  volume  of  work  (either 
sample  inspection  or  100%  inspection)  and  rejection,  repair,  cor¬ 
rection  or  acceptance  according  to  a  predetermined  standard, 

XXV .  Plant  Layout  Analysis 

Plant  layout  analysis  deals  with  the  study  of  the  arrangement  of 
facilities.  Effective  layout  calls  for  a  minimum  of  movement  of 
both  materials  and  person  tel,  nnd  effective  placement  of  lighting, 
neating  and  other  environmental  control  uevices.  Plant  layout 
analysis  may  also  be  defined  as  planning  and  integrating  the  path 
of  the  component  parts  or  activities  of  a  product  or  service  to 
obtain  the  most  effective  and  economical  relationship  between  nan 
ard  equip1  ent.  Principles  to  be  considered  in  planning  a  layout 
are: 


l 

i 


107 


1. 

Overall  Integration. 

7. 

Aesthetics. 

2. 

Minimum  distance  moved. 

8. 

Lighting. 

3. 

Plow. 

9. 

Heating  and  Air 

4. 

Cubic  space  utilization. 

-■ 

conditioning. 

5. 

Safety 

10. 

Noise 

6. 

Flexibility. 

11. 

Comfort. 

O 


(References:  19,  263.) 

Indicates  efficiency  and  effectiveness  relative  to  the  amount  of 
improvements  which  can  be  made  to  obtain  more  effective  and  economical 
relationships  between  people  (working  with  or  using  the  facilities) 
and  the  plant's  physical  facilities.  Layout  charts  can  be  utilized 
to  judge  feasibility  of  alternate  arrangements  (Layout  Chart,  X). 


XXVI .  Organizational  Analysis 


Study  of  the  purpose  for  the  existence  of  an  organization,  its 
functions  and  the  extent  to  which  it  accomplishes  its  functions.  Uses 
organization  charts  to  delineate  functional  areas  of  organization 
(Organization  Chart,  IX).  (References:  96,  126,  146,  204,  205, 

206,  227,  265,  394.) 

Indicates  efficiency  and  effectiveness  of  an  organization  by  one  or 
more  of  a  variety  of  systematic  approaches  (for  example,  abstracts  96, 

126,  146,  204,  206,  227,  394,  Phase  I).  These  studies  measure  efficiency, 
effectiveness  and  performance  by  considering  the  completeness  of  and 
the  degree  of  consistency  between  missions,  goals,  roles  and  outputs 
of  each  unit  and  stratum  within  the  bureaucracy. 
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XXVII.  Research 


Methods  of  gathering  information  by  Interviews,  questionnaires, 
observation  and  surveys  and  utilizing  this  information  to  draw  ob¬ 
jective  conclusions.  This  technique  is  intended  to  include  a  wide 
variety  of  methods  of  gathering  information  not  included  elsewhere 
in  the  glossary  of  techniques,  but  represented  in  the  following 
abstracts:  6,  24,  25,  27,  31,  42,  68,  71,  80,  92,  183,  211,  240,  386, 
396,  397,  421,  432,  Phase  I. 

Indicates  efficiency  and  effectiveness  by  a  wide  variety  of  techniques 
Incorporating  questionnaires,  observations  and  surveys  to  gather 
performance  data  and  by  incorporating  various  techniques  to  process 
these  data  in  order  to  draw  conclusions  about  efficiency,  effectiveness 
and  performance. 


O 
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